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I n writing this book, I was guided by my long-standing experience and interest in teaching 
discrete mathematics. For the student, my purpose was to present material in a precise, read- 
able manner, with the concepts and techniques of discrete mathematics clearly presented and 
demonstrated. My goal was to show the relevance and practicality of discrete mathematics to 
students, who are often skeptical. I wanted to give students studying computer science all of 
the mathematical foundations they need for their future studies. I wanted to give mathematics 
students an understanding of important mathematical concepts together with a sense of why 
these concepts are important for applications. And most importantly, I wanted to accomplish 
these goals without watering down the material. 

For the instructor, my purpose was to design a flexible, comprehensive teaching tool using 
proven pedagogical techniques in mathematics. I wanted to provide instructors with a package 
of materials that they could use to teach discrete mathematics effectively and efficiently in the 
most appropriate manner for their particular set of students. I hope that I have achieved these 
goals. 

I have been extremely gratified by the tremendous success of this text, including its use 
by more than one million students around the world over the last 30 years and its translation 
into many different languages. The many improvements in the eighth edition have been made 
possible by the feedback and suggestions of a large number of instructors and students at many 
of the more than 600 North American schools, and at many universities in different parts of the 
world, where this book has been successfully used. I have been able to significantly improve the 
appeal and effectiveness of this book edition to edition because of the feedback I have received 
and the significant investments that have been made in the evolution of the book. 

This text is designed for a one- or two-term introductory discrete mathematics course taken 
by students in a wide variety of majors, including mathematics, computer science, and engineer- 
ing. College algebra is the only explicit prerequisite, although a certain degree of mathematical 
maturity is needed to study discrete mathematics in a meaningful way. This book has been de- 
signed to meet the needs of almost all types of introductory discrete mathematics courses. It is 
highly flexible and extremely comprehensive. The book is designed not only to be a successful 
textbook, but also to serve as a valuable resource students can consult throughout their studies 
and professional life. 


Goals of a Discrete Mathematics Course 


A discrete mathematics course has more than one purpose. Students should learn a particular 
set of mathematical facts and how to apply them; more importantly, such a course should teach 
students how to think logically and mathematically. To achieve these goals, this text stresses 
mathematical reasoning and the different ways problems are solved. Five important themes are 
interwoven in this text: mathematical reasoning, combinatorial analysis, discrete structures, al- 
gorithmic thinking, and applications and modeling. A successful discrete mathematics course 
should carefully blend and balance all five themes. 


1. Mathematical Reasoning: Students must understand mathematical reasoning in order to read, 
comprehend, and construct mathematical arguments. This text starts with a discussion of 
mathematical logic, which serves as the foundation for the subsequent discussions of methods 
of proof. Both the science and the art of constructing proofs are addressed. The technique of 
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mathematical induction is stressed through many different types of examples of such proofs 
and a careful explanation of why mathematical induction is a valid proof technique. 


2. Combinatorial Analysis: An important problem-solving skill is the ability to count or enu- 
merate objects. The discussion of enumeration in this book begins with the basic techniques 
of counting. The stress is on performing combinatorial analysis to solve counting problems 
and analyze algorithms, not on applying formulae. 


3. Discrete Structures: A course in discrete mathematics should teach students how to work 
with discrete structures, which are the abstract mathematical structures used to represent 
discrete objects and relationships between these objects. These discrete structures include 
sets, permutations, relations, graphs, trees, and finite-state machines. 


4. Algorithmic Thinking: Certain classes of problems are solved by the specification of an 
algorithm. After an algorithm has been described, a computer program can be constructed 
implementing it. The mathematical portions of this activity, which include the specification 
of the algorithm, the verification that it works properly, and the analysis of the computer 
memory and time required to perform it, are all covered in this text. Algorithms are described 
using both English and an easily understood form of pseudocode. 


5. Applications and Modeling: Discrete mathematics has applications to almost every conceiv- 
able area of study. There are many applications to computer science and data networking 
in this text, as well as applications to such diverse areas as chemistry, biology, linguistics, 
geography, business, and the Internet. These applications are natural and important uses of 
discrete mathematics and are not contrived. Modeling with discrete mathematics is an ex- 
tremely important problem-solving skill, which students have the opportunity to develop by 
constructing their own models in some of the exercises. 


Changes in the Eighth Edition 


Although the seventh edition has been an extremely effective text, many instructors have re- 
quested changes to make the book more useful to them. I have devoted a significant amount of 
time and energy to satisfy their requests and I have worked hard to find my own ways to improve 
the book and to keep it up-to-date. 

The eighth edition includes changes based on input from more than 20 formal reviewers, 
feedback from students and instructors, and my insights. The result is a new edition that I ex- 
pect will be a more effective teaching tool. Numerous changes in the eighth edition have been 
designed to help students learn the material. Additional explanations and examples have been 
added to clarify material where students have had difficulty. New exercises, both routine and 
challenging, have been added. Highly relevant applications, including many related to the In- 
ternet, to computer science, and to mathematical biology, have been added. The companion 
website has benefited from extensive development; it now provides extensive tools students can 
use to master key concepts and to explore the world of discrete mathematics. Furthermore, addi- 
tional effective and comprehensive learning and assessment tools are available, complementing 
the textbook. 

I hope that instructors will closely examine this new edition to discover how it might meet 
their needs. Although it is impractical to list all the changes in this edition, a brief list that 
highlights some key changes, listed by the benefits they provide, may be useful. 


Changes in the Eighth Edition 


This new edition of the book includes many enhancements, updates, additions, and edits, all 
designed to make the book a more effective teaching tool for a modern discrete mathematics 
course. Instructors who have used the book previously will notice overall changes that have been 
made throughout the book, as well as specific changes. The most notable revisions are described 
here. 
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Overall Changes 


> 


> 


> 


> 


Exposition has been improved throughout the book with a focus on providing more clarity 
to help students read and comprehend concepts. 


Many proofs have been enhanced by adding more details and explanations, and by re- 
minding the reader of the proof methods used. 


New examples have been added, often to meet needs identified by reviewers or to illus- 
trate new material. Many of these examples are found in the text, but others are available 
only on the companion website. 


Many new exercises, both routine and challenging, address needs identified by in- 
structors or cover new material, while others strengthen and broaden existing exercise 
sets. 


More second and third level heads have been used to break sections into smaller co- 
herent parts, and a new numbering scheme has been used to identify subsections of the 
book. 


The online resources for this book have been greatly expanded, providing extensive sup- 
port for both instructors and students. These resources are described later in the front 
matter. 


Topic Coverage 


> 


Logic Several logical puzzles have been introduced. A new example explains how to 
model the n-queens problem as a satisfiability problem that is both concise and accessible 
to students. 


Set theory Multisets are now covered in the text. (Previously they were introduced in 
the exercises.) 


Algorithms The string matching problem, an important algorithm for many applica- 
tions, including spell checking, key-word searching, string-matching, and computational 
biology, is now discussed. The brute-force algorithm for solving string-matching exer- 
cises is presented. 


Number theory The new edition includes the latest numerical and theoretic discov- 
eries relating to primes and open conjectures about them. The extended Euclidean algo- 
rithm, a one-pass algorithm, is now discussed in the text. (Previously it was covered in 
the exercises.) 


Cryptography The concept of homomorphic encryption, and its importance to cloud 
computing, is now covered. 


Mathematical induction The template for proofs by mathematical induction has 
been expanded. It is now placed in the text before examples of proof by mathematical 
induction. 


Counting methods The coverage of the division rule for counting has been expanded. 


Data mining Association rules—key concepts in data mining—are now discussed 
in the section on n-ary relations. Also, the Jaccard metric, which is used to find the 
distance between two sets and which is used in data mining, is introduced in the 
exercises. 


Graph theory applications A new example illustrates how semantic networks, an 
important structure in artificial intelligence, can be modeled using graphs. 
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> Biographies New biographies of Wiles, Bhaskaracharya, de la Vallée-Poussin, 
Hadamard, Zhang, and Gentry have been added. Existing biographies have been ex- 
panded and updated. This adds diversity by including more historically important Eastern 
mathematicians, major nineteenth and twentieth century researchers, and currently active 
twenty-first century mathematicians and computer scientists. 


Features of the Book 


ACCESSIBILITY This text has proven to be easily read and understood by many begin- 
ning students. There are no mathematical prerequisites beyond college algebra for almost all 
the contents of the text. Students needing extra help will find tools on the companion website 
for bringing their mathematical maturity up to the level of the text. The few places in the book 
where calculus is referred to are explicitly noted. Most students should easily understand the 
pseudocode used in the text to express algorithms, regardless of whether they have formally 
studied programming languages. There is no formal computer science prerequisite. 

Each chapter begins at an easily understood and accessible level. Once basic mathematical 
concepts have been carefully developed, more difficult material and applications to other areas 
of study are presented. 


FLEXIBILITY This text has been carefully designed for flexible use. The dependence 
of chapters on previous material has been minimized. Each chapter is divided into sections of 
approximately the same length, and each section is divided into subsections that form natural 
blocks of material for teaching. Instructors can easily pace their lectures using these blocks. 


WRITING STYLE The writing style in this book is direct and pragmatic. Precise math- 
ematical language is used without excessive formalism and abstraction. Care has been taken to 
balance the mix of notation and words in mathematical statements. 


MATHEMATICAL RIGOR AND PRECISION All definitions and theorems in this text 
are stated extremely carefully so that students will appreciate the precision of language and 
rigor needed in mathematics. Proofs are motivated and developed slowly; their steps are all 
carefully justified. The axioms used in proofs and the basic properties that follow from them 
are explicitly described in an appendix, giving students a clear idea of what they can assume in 
a proof. Recursive definitions are explained and used extensively. 


WORKED EXAMPLES Over 800 examples are used to illustrate concepts, relate dif- 
ferent topics, and introduce applications. In most examples, a question is first posed, then its 
solution is presented with the appropriate amount of detail. 


APPLICATIONS The applications included in this text demonstrate the utility of discrete 
mathematics in the solution of real-world problems. This text includes applications to a wide 
variety of areas, including computer science, data networking, psychology, chemistry, engineer- 
ing, linguistics, biology, business, and the Internet. 


ALGORITHMS Results in discrete mathematics are often expressed in terms of algo- 
rithms; hence, key algorithms are introduced in most chapters of the book. These algorithms 
are expressed in words and in an easily understood form of structured pseudocode, which is 
described and specified in Appendix 3. The computational complexity of the algorithms in the 
text is also analyzed at an elementary level. 


HISTORICAL INFORMATION The background of many topics is succinctly described 
in the text. Brief biographies of 89 mathematicians and computer scientists are included as 
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footnotes. These biographies include information about the lives, careers, and accomplishments 
of these important contributors to discrete mathematics, and images, when available, are dis- 
played. 

In addition, numerous historical footnotes are included that supplement the historical in- 
formation in the main body of the text. Efforts have been made to keep the book up-to-date by 
reflecting the latest discoveries. 


KEY TERMS AND RESULTS A list of key terms and results follows each chapter. The 
key terms include only the most important that students should learn, and not every term defined 
in the chapter. 


EXERCISES There are over 4200 exercises in the text, with many different types of ques- 
tions posed. There is an ample supply of straightforward exercises that develop basic skills, a 
large number of intermediate exercises, and many challenging exercises. Exercises are stated 
clearly and unambiguously, and all are carefully graded for level of difficulty. Exercise sets con- 
tain special discussions that develop new concepts not covered in the text, enabling students to 
discover new ideas through their own work. 

Exercises that are somewhat more difficult than average are marked with a single star, *; 
those that are much more challenging are marked with two stars, **. Exercises whose solutions 
require calculus are explicitly noted. Exercises that develop results used in the text are clearly 
identified with the right pointing hand symbol, t=. Answers or outlined solutions to all odd- 
numbered exercises are provided at the back of the text. The solutions include proofs in which 
most of the steps are clearly spelled out. 


REVIEW QUESTIONS A set of review questions is provided at the end of each chapter. 
These questions are designed to help students focus their study on the most important concepts 
and techniques of that chapter. To answer these questions students need to write long answers, 
rather than just perform calculations or give short replies. 


SUPPLEMENTARY EXERCISE SETS Each chapter is followed by a rich and varied 
set of supplementary exercises. These exercises are generally more difficult than those in the 
exercise sets following the sections. The supplementary exercises reinforce the concepts of the 
chapter and integrate different topics more effectively. 


COMPUTER PROJECTS Each chapter is followed by a set of computer projects. The 
approximately 150 computer projects tie together what students may have learned in computing 
and in discrete mathematics. Computer projects that are more difficult than average, from both 
a mathematical and a programming point of view, are marked with a star, and those that are 
extremely challenging are marked with two stars. 


COMPUTATIONS AND EXPLORATIONS A set of computations and explorations is 
included at the conclusion of each chapter. These exercises (approximately 120 in total) are de- 
signed to be completed using existing software tools, such as programs that students or instruc- 
tors have written or mathematical computation packages such as Maple™ or Mathematica™. 
Many of these exercises give students the opportunity to uncover new facts and ideas through 
computation. (Some of these exercises are discussed in the Exploring Discrete Mathematics 
companion workbooks available online.) 


WRITING PROJECTS Each chapter is followed by a set of writing projects. To do these 
projects students need to consult the mathematical literature. Some of these projects are his- 
torical in nature and may involve looking up original sources. Others are designed to serve as 
gateways to new topics and ideas. All are designed to expose students to ideas not covered in 
depth in the text. These projects tie mathematical concepts together with the writing process and 
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help expose students to possible areas for future study. (Suggested references for these projects 
can be found online or in the printed Student’s Solutions Guide.) 


APPENDICES There are three appendices to the text. The first introduces axioms for real 
numbers and the positive integers, and illustrates how facts are proved directly from these ax- 
ioms. The second covers exponential and logarithmic functions, reviewing some basic material 
used heavily in the course. The third specifies the pseudocode used to describe algorithms in 
this text. 


SUGGESTED READINGS A list of suggested readings for the overall book and for each 
chapter is provided after the appendices. These suggested readings include books at or below 
the level of this text, more difficult books, expository articles, and articles in which discoveries 
in discrete mathematics were originally published. Some of these publications are classics, pub- 
lished many years ago, while others have been published in the last few years. These suggested 
readings are complemented by the many links to valuable resources available on the web that 
can be found on the website for this book. 


How to Use This Book 


This text has been carefully written and constructed to support discrete mathematics courses 
at several levels and with differing foci. The following table identifies the core and optional 
sections. An introductory one-term course in discrete mathematics at the sophomore level can 
be based on the core sections of the text, with other sections covered at the discretion of the 
instructor. A two-term introductory course can include all the optional mathematics sections in 
addition to the core sections. A course with a strong computer science emphasis can be taught 
by covering some or all of the optional computer science sections. Instructors can find sample 
syllabi for a wide range of discrete mathematics courses and teaching suggestions for using each 
section of the text can be found in the /nstructor’s Resource Guide available on the website for 
this book. 


Optional CS | Optional Math 


1.1-1.8 (as needed) 
2.1-2.4, 2.6 (as needed) 

3.1-3.3 (as needed) 
4.1—4.4 (as needed) 


12.1-12.4 
13.1-13.5 


Instructors using this book can adjust the level of difficulty of their course by choosing 
either to cover or to omit the more challenging examples at the end of sections, as well as 
the more challenging exercises. The chapter dependency chart shown here displays the strong 
dependencies. A star indicates that only relevant sections of the chapter are needed for study 
of a later chapter. Weak dependencies have been ignored. More details can be found in the 
Instructor’s Resource Guide. 


Ancillaries 
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STUDENT’S SOLUTIONS GUIDE This student manual, available separately, contains 
full solutions to all odd-numbered exercises in the exercise sets. These solutions explain why 
a particular method is used and why it works. For some exercises, one or two other possible 
approaches are described to show that a problem can be solved in several different ways. Sug- 
gested references for the writing projects found at the end of each chapter are also included in 
this volume. Also included are a guide to writing proofs and an extensive description of com- 
mon mistakes students make in discrete mathematics, plus sample tests and a sample crib sheet 
for each chapter designed to help students prepare for exams. 


INSTRUCTOR’S RESOURCE GUIDE This manual, available on the website and in 
printed form by request for instructors, contains full solutions to even-numbered exercises in 
the text. Suggestions on how to teach the material in each chapter of the book are provided, 
including the points to stress in each section and how to put the material into perspective. It 
also offers sample tests for each chapter and a test bank containing over 1500 exam questions to 
choose from. Answers to all sample tests and test bank questions are included. Finally, sample 
syllabi are presented for courses with differing emphases and student ability levels. 
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Online Resources 


Bz effort has been devoted to producing valuable web resources for this book. In- 
structors should make a special effort to explore these resources to identify those they feel 
will help their students learn and explore discrete mathematics. These resources are available in 
the Online Learning Center, which is available to all students and instructors, and the Connect 
Site, designed for interactive instruction, which instructors can choose to use. To use Connect, 
students purchase online access for a specific time period. 


The Online Learning Center 
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Extra » 
Examples 


Demo » 


Assessment > 


Links >» 


The Online Learning Center (OLC), accessible at www.mhhe.com/rosen, includes an Informa- 
tion Center, a Student Site, and an Instructor Site. Key features of each area are described here. 


0.1.1 The Information Center 


The Information Center contains basic information about the book including the expanded table 
of contents (including subsection heads), the preface, descriptions of the ancillaries, and a sam- 
ple chapter. It also provides a link that can be used to submit errata reports and other feedback 
about the book. 


0.1.2 Student Site 


The Student Site contains a wealth of resources available for student use, including the follow- 
ing, tied into the text wherever the special icons displayed below are found in the text: 


> Extra Examples You can find a large number of additional examples on the site, cov- 
ering all chapters of the book. These examples are concentrated in areas where students 
often ask for additional material. Although most of these examples amplify the basic 
concepts, more-challenging examples can also be found here. Many new extra examples 
have been recently added for the eighth edition. Each icon in the book corresponds to 
one or more extra examples on the website. 

> Interactive Demonstration Applets These applets enable you to interactively explore 
how important algorithms work, and are tied directly to material in the text with linkages 
to examples and exercises. Additional resources are provided on how to use and apply 
these applets. 

> Self Assessments These interactive guides help you assess your understanding of 14 
key concepts, providing a question bank where each question includes a brief tutorial 
followed by a multiple-choice question. If you select an incorrect answer, advice is pro- 
vided to help you understand your error. Using these Self Assessments, you should be 
able to diagnose your problems and find appropriate help. 

> Web Resources Guide This guide provides annotated links to hundreds of external 
websites containing relevant material such as historical and biographical information, 
puzzles and problems, discussions, applets, programs, and more. These links are keyed 
to the text by page number. 
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Additional resources in the Student Site include: 


> Exploring Discrete Mathematics This ancillary provides help for using a computer alge- 
bra system to do a wide range of computations in discrete mathematics. Each chapter provides 
a description of relevant functions in the computer algebra system and how they are used, pro- 
grams to carry out computations in discrete mathematics, examples, and exercises that can be 
worked using this computer algebra system. Two versions, Exploring Discrete Mathematics 
with Maple™ and Exploring Discrete Mathematics with Mathematica™, are available. 


> Applications of Discrete Mathematics This ancillary contains 24 chapters—each with 
its own set of exercises—presenting a wide variety of interesting and important applications 
covering three general areas in discrete mathematics: discrete structures, combinatorics, and 
graph theory. These applications are ideal for supplementing the text or for independent study. 


> A Guide to Proof-Writing This guide provides additional help for writing proofs, a skill 
that many students find difficult to master. By reading this guide at the beginning of the 
course and periodically thereafter when proof writing is required, you will be rewarded as 
your proof-writing ability grows. (Also available in the Student’s Solutions Guide.) 


> Common Mistakes in Discrete Mathematics This guide includes a detailed list of com- 
mon misconceptions that students of discrete mathematics often have and the kinds of errors 
they tend to make. You are encouraged to review this list from time to time to help avoid 
these common traps. (Also available in the Student’s Solutions Guide.) 


> Advice on Writing Projects This guide offers helpful hints and suggestions for the Writing 
Projects in the text, including an extensive bibliography of helpful books and articles for 
research, discussion of various resources available in print and online, tips on doing library 
research, and suggestions on how to write well. (Also available in the Student’s Solutions 
Guide.) 


0.1.3 Instructor Site 


This part of the website provides access to all of the resources on the Student Site, as well as 
these resources for instructors: 


> Suggested Syllabi Detailed course outlines are shown, offering suggestions for 
courses with different emphases and different student backgrounds and ability levels. 

> Teaching Suggestions This guide contains detailed teaching suggestions for instruc- 
tors, including chapter overviews for the entire text, detailed remarks on each section, 
and comments on the exercise sets. 

> Printable Tests Printable tests are offered in TeX and Word format for every chapter, 
and can be customized by instructors. 

> PowerPoint Lecture Slides and PowerPoint Figures and Tables An extensive col- 
lection of PowerPoint lecture notes for all chapters of the text are provided for instructor 
use. In addition, images of all figures and tables from the text are provided as PowerPoint 
slides. 


0.1.4 Connect 


A comprehensive online learning package has been developed in conjunction with the text. A 
high-level description of this site will be provided here. Interested instructors and students can 
find out more about Connect from McGraw-Hill Higher Education. When instructors choose to 
use this option, students in their classes must obtain access to Connect for this text, either by 
purchasing a copy of the textbook that also includes access privileges or by purchasing access 
only with the option of buying a loose-leaf version of the textbook. 
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Instructors who adopt Connect have access to a full-featured course management system. 
Course management capabilities are provided that allow instructors to create assignments, au- 
tomatically assign and grade homework, quiz, and test questions from a bank of questions tied 
directly to the text, create and edit their own questions, manage course announcements and due 
dates, and track student progress. 

Instructors can create their own assignments using Connect. They select the particular ex- 
ercises from each section of the book that they want to assign. They can also assign chapters 
from the SmartBook version of the text, which provides an adaptive learning environment. Stu- 
dents have access to the interactive version of the textbook, the online homework exercises, and 
SmartBook. 


Interactive Textbook Students have access to an easy-to-use interactive version of the textbook 
when they use Connect. The interactive site provides the full content of the text, as well as the 
many extra resources that enrich the book. The resources include extra examples, interactive 
demonstrations, and self-assessments. 


Homework and Learning Management Solution An extensive learning management solution 
has been developed that instructors can use to construct homework assignments. Approximately 
800 online questions are available, including questions from every section of the text. These 
questions are tied to the most commonly assigned exercises in the book. 

These online questions have been constructed to support the same objectives as the corre- 
sponding written homework questions. This challenge has been met by stretching the capabili- 
ties of different question types supported by the Connect platform. 


SmartBook Connect also provides another extended online version of the text in the McGraw- 
Hill SmartBook platform. The SmartBook version of the text includes a set of objectives for each 
chapter of the text. A collection of questions, called probes, is provided to assess student un- 
derstanding of each objective. Students are directed to the appropriate part of the text to review 
the material needed for each of these objectives. SmartBook provides an adaptive learning en- 
vironment; it selects probes for students based on their performance answering earlier probes. 
Instructors can assign SmartBook as assignments or can have their students use SmartBook as 
a learning tool. 


To the Student 


W is discrete mathematics ? Discrete mathematics is the part of mathematics devoted 
to the study of discrete objects. (Here discrete means consisting of distinct or uncon- 
nected elements.) The kinds of problems solved using discrete mathematics include: 


> How many ways are there to choose a valid password on a computer system? 
> What is the probability of winning a lottery? 

> Is there a link between two computers in a network? 

> How can I identify spam e-mail messages? 

> How can I encrypt a message so that no unintended recipient can read it? 

> What is the shortest path between two cities using a transportation system? 

> How can a list of integers be sorted so that the integers are in increasing order? 
> How many steps are required to do such a sorting? 

> How can it be proved that a sorting algorithm correctly sorts a list? 

> How can a circuit that adds two integers be designed? 


> How many valid Internet addresses are there? 


You will learn the discrete structures and techniques needed to solve problems such as these. 

More generally, discrete mathematics is used whenever objects are counted, when relation- 
ships between finite (or countable) sets are studied, and when processes involving a finite num- 
ber of steps are analyzed. A key reason for the growth in the importance of discrete mathematics 
is that information is stored and manipulated by computing machines in a discrete fashion. 


WHY STUDY DISCRETE MATHEMATICS? There are several important reasons for 
studying discrete mathematics. First, through this course you can develop your mathematical 
maturity: that is, your ability to understand and create mathematical arguments. You will not 
get very far in your studies in the mathematical sciences without these skills. 

Second, discrete mathematics is the gateway to more advanced courses in all parts of the 
mathematical sciences. Discrete mathematics provides the mathematical foundations for many 
computer science courses, including data structures, algorithms, database theory, automata the- 
ory, formal languages, compiler theory, computer security, and operating systems. Students find 
these courses much more difficult when they have not had the appropriate mathematical foun- 
dations from discrete mathematics. One student sent me an e-mail message saying that she used 
the contents of this book in every computer science course she took! 

Math courses based on the material studied in discrete mathematics include logic, set theory, 
number theory, linear algebra, abstract algebra, combinatorics, graph theory, and probability 
theory (the discrete part of the subject). 

Also, discrete mathematics contains the necessary mathematical background for solv- 
ing problems in operations research (including discrete optimization), chemistry, engineering, 
biology, and so on. In the text, we will study applications to some of these areas. 

Many students find their introductory discrete mathematics course to be significantly more 
challenging than courses they have previously taken. One reason for this is that one of the pri- 
mary goals of this course is to teach mathematical reasoning and problem solving, rather than a 
discrete set of skills. The exercises in this book are designed to reflect this goal. Although there 
are plenty of exercises in this text similar to those addressed in the examples, a large percentage 
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of the exercises require original thought. This is intentional. The material discussed in the text 
provides the tools needed to solve these exercises, but your job is to successfully apply these 
tools using your own creativity. One of the primary goals of this course is to learn how to attack 
problems that may be somewhat different from any you may have previously seen. Unfortu- 
nately, learning how to solve only particular types of exercises is not sufficient for success in 
developing the problem-solving skills needed in subsequent courses and professional work. This 
text addresses many different topics, but discrete mathematics is an extremely diverse and large 
area of study. One of my goals as an author is to help you develop the skills needed to master 
the additional material you will need in your own future pursuits. 

Finally, discrete mathematics is an excellent environment in which to learn how to read and 
write mathematical proofs. In addition to explicit material on proofs in Chapter 1 and Chapter 5, 
this textbook contains throughout many proofs of theorems and many exercises asking the stu- 
dent to prove statements. This not only deepens one’s understanding of the subject matter but is 
also valuable preparation for more advanced courses in mathematics and theoretical computer 
science. 


THE EXERCISES I would like to offer some advice about how you can best learn discrete 
mathematics (and other subjects in the mathematical and computing sciences). You will learn 
the most by actively working exercises. I suggest that you solve as many as you possibly can. 
After working the exercises your instructor has assigned, I encourage you to solve additional 
exercises such as those in the exercise sets following each section of the text and in the supple- 
mentary exercises at the end of each chapter. (Note the key explaining the markings preceding 
exercises.) 


Key to the Exercises 


no marking A routine exercise 


= A difficult exercise 


üi An extremely challenging exercise 


t= An exercise containing a result used in the book (Table 1 on the 
following page shows where these exercises are used.) 


(Requires calculus) An exercise whose solution requires the use of limits or concepts 
from differential or integral calculus 


The best approach is to try exercises yourself before you consult the answer section at the 
end of this book. Note that the odd-numbered exercise answers provided in the text are answers 
only and not full solutions; in particular, the reasoning required to obtain answers is omitted in 
these answers. The Student’s Solutions Guide, available separately, provides complete, worked 
solutions to all odd-numbered exercises in this text. When you hit an impasse trying to solve an 
odd-numbered exercise, I suggest you consult the Student’s Solutions Guide and look for some 
guidance as to how to solve the problem. The more work you do yourself rather than passively 
reading or copying solutions, the more you will learn. The answers and solutions to the even- 
numbered exercises are intentionally not available from the publisher; ask your instructor if you 
have trouble with these. 


WEB RESOURCES All users of the book are able to access the online resources acces- 
sible via the Online Learning Center (OLC) for the book. You will find many Extra Exam- 
ples designed to clarify key concepts, Self Assessments for gauging how well you understand 
core topics, Interactive Demonstrations that explore key algorithms and other concepts, a Web 
Resources Guide containing an extensive selection of links to external sites relevant to the 
world of discrete mathematics, extra explanations and practice to help you master core con- 
cepts, added instruction on writing proofs and on avoiding common mistakes in discrete mathe- 
matics, in-depth discussions of important applications, and guidance on utilizing Maple™ and 
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TABLE 1 Hand-Icon Exercises and Where They Are Used 


Mathematica™ software to explore the computational aspects of discrete mathematics. Places 
in the text where these additional online resources are available are identified in the margins by 
special icons. For more details on these and other online resources, see the description of the 
companion website immediately preceding this “To the Student” message. 


THE VALUE OF THIS BOOK My intention is to make your substantial investment in this 
text an excellent value. The book, the associated ancillaries, and companion website have taken 
many years of effort to develop and refine. I am confident that most of you will find that the text 
and associated materials will help you master discrete mathematics, just as so many previous 
students have. Even though it is likely that you will not cover some chapters in your current 
course, you should find it helpful—as many other students have—to read the relevant sections 
of the book as you take additional courses. Most of you will return to this book as a useful tool 
throughout your future studies, especially for those of you who continue in computer science, 
mathematics, and engineering. I have designed this book to be a gateway for future studies and 
explorations, and to be comprehensive reference, and I wish you luck as you begin your journey. 


Kenneth H. Rosen 
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e Foundations: 
Logic and Proofs 


T= rules of logic specify the meaning of mathematical statements. For instance, these 
rules help us understand and reason with statements such as “There exists an integer 
that is not the sum of two squares” and “For every positive integer n, the sum of the posi- 
tive integers not exceeding n is n(n + 1)/2.” Logic is the basis of all mathematical reasoning, 
and of all automated reasoning. It has practical applications to the design of computing ma- 
chines, to the specification of systems, to artificial intelligence, to computer programming, to 
programming languages, and to other areas of computer science, as well as to many other fields 
of study. 

To understand mathematics, we must understand what makes up a correct mathematical 
argument, that is, a proof. Once we prove a mathematical statement is true, we call it a theorem. 
A collection of theorems on a topic organize what we know about this topic. To learn a math- 
ematical topic, a person needs to actively construct mathematical arguments on this topic, and 
not just read exposition. Moreover, knowing the proof of a theorem often makes it possible to 
modify the result to fit new situations. 

Everyone knows that proofs are important throughout mathematics, but many people find 
it surprising how important proofs are in computer science. In fact, proofs are used to verify 
that computer programs produce the correct output for all possible input values, to show that 
algorithms always produce the correct result, to establish the security of a system, and to create 
artificial intelligence. Furthermore, automated reasoning systems have been created to allow 
computers to construct their own proofs. 

In this chapter, we will explain what makes up a correct mathematical argument and intro- 
duce tools to construct these arguments. We will develop an arsenal of different proof methods 
that will enable us to prove many different types of results. After introducing many different 
methods of proof, we will introduce several strategies for constructing proofs. We will intro- 
duce the notion of a conjecture and explain the process of developing mathematics by studying 
conjectures. 


Propositional Logic 


1.1.1 Introduction 


The rules of logic give precise meaning to mathematical statements. These rules are used to dis- 
tinguish between valid and invalid mathematical arguments. Because a major goal of this book 
is to teach the reader how to understand and how to construct correct mathematical arguments, 
we begin our study of discrete mathematics with an introduction to logic. 

Besides the importance of logic in understanding mathematical reasoning, logic has numer- 
ous applications to computer science. These rules are used in the design of computer circuits, 
the construction of computer programs, the verification of the correctness of programs, and in 
many other ways. Furthermore, software systems have been developed for constructing some, 
but not all, types of proofs automatically. We will discuss these applications of logic in this and 
later chapters. 
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1 / The Foundations: Logic and Proofs 


1.1.2 Propositions 


Our discussion begins with an introduction to the basic building blocks of logic—propositions. 
A proposition is a declarative sentence (that is, a sentence that declares a fact) that is either 
true or false, but not both. 


EXAMPLE 1 All the following declarative sentences are propositions. 


Extra » 1. Washington, D.C., is the capital of the United States of America. 
Examples 2. Toronto is the capital of Canada. 
3. 14+1=2. 
4.242=3. 
Propositions 1 and 3 are true, whereas 2 and 4 are false. < 


Some sentences that are not propositions are given in Example 2. 


EXAMPLE 2 Consider the following sentences. 


1. What time is it? 

2. Read this carefully. 
3.x+1=2. 
4.x+y=z 


Sentences 1 and 2 are not propositions because they are not declarative sentences. Sentences 3 
and 4 are not propositions because they are neither true nor false. Note that each of sentences 3 
and 4 can be turned into a proposition if we assign values to the variables. We will also discuss 
other ways to turn sentences such as these into propositions in Section 1.4. < 


We use letters to denote propositional variables (or sentential variables), that is, variables 
that represent propositions, just as letters are used to denote numerical variables. The conven- 
tional letters used for propositional variables are p, q, r, s, ... . The truth value of a proposition 


ARISTOTLE (384 B.C.E.-322 B.C.E.) Aristotle was born in Stagirus (Stagira) in northern Greece. His father 
was the personal physician of the King of Macedonia. Because his father died when Aristotle was young, 
Aristotle could not follow the custom of following his father’s profession. Aristotle became an orphan at a 
young age when his mother also died. His guardian who raised him taught him poetry, rhetoric, and Greek. 
At the age of 17, his guardian sent him to Athens to further his education. Aristotle joined Plato’s Academy, 
where for 20 years he attended Plato’s lectures, later presenting his own lectures on rhetoric. When Plato died in 
347 B.C.E., Aristotle was not chosen to succeed him because his views differed too much from those of Plato. 
| Instead, Aristotle joined the court of King Hermeas where he remained for three years, and married the niece 
Source: National Library of Of the King. When the Persians defeated Hermeas, Aristotle moved to Mytilene and, at the invitation of King 
Medicine Philip of Macedonia, he tutored Alexander, Philip’s son, who later became Alexander the Great. Aristotle tutored 
Alexander for five years and after the death of King Philip, he returned to Athens and set up his own school, 
called the Lyceum. 

Aristotle’s followers were called the peripatetics, which means “to walk about,” because Aristotle often walked around as he 
discussed philosophical questions. Aristotle taught at the Lyceum for 13 years where he lectured to his advanced students in the 
morning and gave popular lectures to a broad audience in the evening. When Alexander the Great died in 323 B.C.E., a backlash against 
anything related to Alexander led to trumped-up charges of impiety against Aristotle. Aristotle fled to Chalcis to avoid prosecution. 
He only lived one year in Chalcis, dying of a stomach ailment in 322 B.C.E. 

Aristotle wrote three types of works: those written for a popular audience, compilations of scientific facts, and systematic 
treatises. The systematic treatises included works on logic, philosophy, psychology, physics, and natural history. Aristotle’s writings 
were preserved by a student and were hidden in a vault where a wealthy book collector discovered them about 200 years later. They 
were taken to Rome, where they were studied by scholars and issued in new editions, preserving them for posterity. 
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1.1 Propositional Logic 3 


is true, denoted by T, if it is a true proposition, and the truth value of a proposition is false, de- 
noted by F, if it is a false proposition. Propositions that cannot be expressed in terms of simpler 
propositions are called atomic propositions. 

The area of logic that deals with propositions is called the propositional calculus or propo- 
sitional logic. It was first developed systematically by the Greek philosopher Aristotle more than 
2300 years ago. 

We now turn our attention to methods for producing new propositions from those that we 
already have. These methods were discussed by the English mathematician George Boole in 
1854 in his book The Laws of Thought. Many mathematical statements are constructed by com- 
bining one or more propositions. New propositions, called compound propositions, are formed 
from existing propositions using logical operators. 


Let p be a proposition. The negation of p, denoted by =p (also denoted by p), is the statement 
“Tt is not the case that p.” 


The proposition =p is read “not p.” The truth value of the negation of p, ap, is the opposite 
of the truth value of p. 


Remark: The notation for the negation operator is not standardized. Although =p and p are the 
most common notations used in mathematics to express the negation of p, other notations you 
might see are ~p, —p, p’, Np, and !p. 


Find the negation of the proposition 
“Michael’s PC runs Linux” 


and express this in simple English. 


Solution: The negation is 
“Tt is not the case that Michael’s PC runs Linux.” 
This negation can be more simply expressed as 


“Michael’s PC does not run Linux.” < 


Find the negation of the proposition 
“Vandana’s smartphone has at least 32 GB of memory” 


and express this in simple English. 


Solution: The negation is 

“Tt is not the case that Vandana’s smartphone has at least 32 GB of memory.” 
This negation can also be expressed as 

“Vandana’s smartphone does not have at least 32 GB of memory” 
or even more simply as 


“Vandana’s smartphone has less than 32 GB of memory.” < 
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Table 1 displays the truth table for the negation of a proposition p. This table has a row for 
each of the two possible truth values of p. Each row shows the truth value of =p corresponding 
to the truth value of p for this row. 

The negation of a proposition can also be considered the result of the operation of the 
negation operator on a proposition. The negation operator constructs a new proposition from 
a single existing proposition. We will now introduce the logical operators that are used to form 
new propositions from two or more existing propositions. These logical operators are also called 
connectives. 


Let p and q be propositions. The conjunction of p and q, denoted by p A q, is the proposition 
“p and q.” The conjunction p A q is true when both p and q are true and is false otherwise. 


Table 2 displays the truth table of p A q. This table has a row for each of the four possible 
combinations of truth values of p and q. The four rows correspond to the pairs of truth values 
TT, TF, FT, and FF, where the first truth value in the pair is the truth value of p and the second 
truth value is the truth value of q. 

Note that in logic the word “but” sometimes is used instead of “and” in a conjunction. For 
example, the statement “The sun is shining, but it is raining” is another way of saying “The sun 
is shining and it is raining.” (In natural language, there is a subtle difference in meaning between 
“and” and “but”; we will not be concerned with this nuance here.) 


Find the conjunction of the propositions p and q where p is the proposition “Rebecca’s PC has 
more than 16 GB free hard disk space” and q is the proposition “The processor in Rebecca’s PC 
runs faster than 1 GHz.” 


Solution: The conjunction of these propositions, p A q, is the proposition “Rebecca’s PC has 
more than 16 GB free hard disk space, and the processor in Rebecca’s PC runs faster than 
1 GHz.” This conjunction can be expressed more simply as “Rebecca’s PC has more than 
16 GB free hard disk space, and its processor runs faster than 1 GHz.” For this conjunction 
to be true, both conditions given must be true. It is false when one or both of these conditions 
are false. < 


Let p and q be propositions. The disjunction of p and q, denoted by p V q, is the proposition 
“p or q.” The disjunction p V q is false when both p and q are false and is true otherwise. 


Table 3 displays the truth table for p V q. 


TABLE 2 The Truth Table for TABLE 3 The Truth Table for 
the Conjunction of Two the Disjunction of Two 
Propositions. Propositions. 
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The use of the connective or in a disjunction corresponds to one of the two ways the word 
or is used in English, namely, as an inclusive or. A disjunction is true when at least one of the 
two propositions is true. That is, p V q is true when both p and q are true or when exactly one 
of p and q is true. 


Translate the statement “Students who have taken calculus or introductory computer science can 
take this class” in a statement in propositional logic using the propositions p: “A student who 
has taken calculus can take this class” and q: “A student who has taken introductory computer 
science can take this class.” 


Solution: We assume that this statement means that students who have taken both calculus and 
introductory computer science can take the class, as well as the students who have taken only 
one of the two subjects. Hence, this statement can be expressed as p V q, the inclusive or, or 
disjunction, of p and q. < 


What is the disjunction of the propositions p and q, where p and q are the same propositions as 
in Example 5? 


Solution: The disjunction of p and q, p V q, is the proposition 


“Rebecca’s PC has at least 16 GB free hard disk space, or the processor in Rebecca’s PC 
runs faster than 1 GHz.” 


This proposition is true when Rebecca’s PC has at least 16 GB free hard disk space, when the 
PC’s processor runs faster than 1 GHz, and when both conditions are true. It is false when both 
of these conditions are false, that is, when Rebecca’s PC has less than 16 GB free hard disk 
space and the processor in her PC runs at 1 GHz or slower. < 


Besides its use in disjunctions, the connective or is also used to express an exclusive or. 
Unlike the disjunction of two propositions p and q, the exclusive or of these two propositions is 
true when exactly one of p and q is true; it is false when both p and q are true (and when both 
are false). 


Let p and q be propositions. The exclusive or of p and q, denoted by p @ q (or p XOR q), is 
the proposition that is true when exactly one of p and q is true and is false otherwise. 


GEORGE BOOLE (1815-1864) George Boole, the son of a cobbler, was born in Lincoln, England, in 
November 1815. Because of his family’s difficult financial situation, Boole struggled to educate himself while 
supporting his family. Nevertheless, he became one of the most important mathematicians of the 1800s. Al- 
though he considered a career as a clergyman, he decided instead to go into teaching, and soon afterward 
opened a school of his own. In his preparation for teaching mathematics, Boole—unsatisfied with textbooks 
of his day—decided to read the works of the great mathematicians. While reading papers of the great French 


| mathematician Lagrange, Boole made discoveries in the calculus of variations, the branch of analysis dealing 


with finding curves and surfaces by optimizing certain parameters. 

In 1848 Boole published The Mathematical Analysis of Logic, the first of his contributions to 
symbolic logic. In 1849 he was appointed professor of mathematics at Queen’s College in Cork, 
Ireland. In 1854 he published The Laws of Thought, his most famous work. In this book, Boole 


introduced what is now called Boolean algebra in his honor. Boole wrote textbooks on differential equations and on difference 
equations that were used in Great Britain until the end of the nineteenth century. Boole married in 1855; his wife was the niece of 
the professor of Greek at Queen’s College. In 1864 Boole died from pneumonia, which he contracted as a result of keeping a lecture 
engagement even though he was soaking wet from a rainstorm. 
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The truth table for the exclusive or of two propositions is displayed in Table 4. 


Let p and q be the propositions that state “A student can have a salad with dinner” and “A student 
can have soup with dinner,” respectively. What is p ® q, the exclusive or of p and q? 


Solution: The exclusive or of p and q is the statement that is true when exactly one of p and 
q is true. That is, p ® q is the statement “A student can have soup or salad, but not both, with 
dinner.” Note that this is often stated as “A student can have soup or a salad with dinner,” without 
explicitly stating that taking both is not permitted. < 


Express the statement “I will use all my savings to travel to Europe or to buy an electric car” in 
propositional logic using the statement p: “I will use all my savings to travel to Europe” and the 
statement q: “I will use all my savings to buy an electric car.” 


Solution: To translate this statement, we first note that the or in this statement must be an ex- 
clusive or because this student can either use all his or her savings to travel to Europe or use all 
these savings to buy an electric car, but cannot both go to Europe and buy an electric car. (This 
is clear because either option requires all his savings.) Hence, this statement can be expressed 


asp @ q. < 


1.1.3 Conditional Statements 


We will discuss several other important ways in which propositions can be combined. 


Let p and q be propositions. The conditional statement p > q is the proposition “if p, then 
q.” The conditional statement p > q is false when p is true and q is false, and true otherwise. 
In the conditional statement p > q, p is called the hypothesis (or antecedent or premise) and 
q is called the conclusion (or consequence). 


The statement p > q is called a conditional statement because p —> q asserts that q is true 
on the condition that p holds. A conditional statement is also called an implication. 

The truth table for the conditional statement p — q is shown in Table 5. Note that the state- 
ment p > q is true when both p and q are true and when p is false (no matter what truth value 
q has). 


TABLE 4 The Truth Table for TABLE 5 The Truth Table for 
the Exclusive Or of Two the Conditional Statement 
Propositions. 


You might have trouble 
understanding how 
“unless” is used in 
conditional statements 
unless you read this 
paragraph carefully. 
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Because conditional statements play such an essential role in mathematical reasoning, a va- 
riety of terminology is used to express p > q. You will encounter most if not all of the following 
ways to express this conditional statement: 


“if p, then q” “p implies q” 

“if p, q” “p only if q” 

“p is sufficient for q” “a sufficient condition for q is p” 
“q if p” “q whenever p” 

“q when p” “q is necessary for p” 

“a necessary condition for p is q” “q follows from p” 

“q unless ap” “q provided that p” 


A useful way to understand the truth value of a conditional statement is to think of an obli- 
gation or a contract. For example, the pledge many politicians make when running for office is 


“If I am elected, then I will lower taxes.” 


If the politician is elected, voters would expect this politician to lower taxes. Furthermore, if 
the politician is not elected, then voters will not have any expectation that this person will lower 
taxes, although the person may have sufficient influence to cause those in power to lower taxes. 
It is only when the politician is elected but does not lower taxes that voters can say that the 
politician has broken the campaign pledge. This last scenario corresponds to the case when p is 
true but q is false in p > q. 

Similarly, consider a statement that a professor might make: 


“If you get 100% on the final, then you will get an A.” 


If you manage to get 100% on the final, then you would expect to receive an A. If you do not 
get 100%, you may or may not receive an A depending on other factors. However, if you do get 
100%, but the professor does not give you an A, you will feel cheated. 


Remark: Because some of the different ways to express the implication p implies q can be 
confusing, we will provide some extra guidance. To remember that “p only if q” expresses the 
same thing as “if p, then q,” note that “p only if q” says that p cannot be true when q is not true. 
That is, the statement is false if p is true, but q is false. When p is false, q may be either true or 
false, because the statement says nothing about the truth value of q. 

For example, suppose your professor tells you 


“You can receive an A in the course only if your score on the final is at least 90%.” 


Then, if you receive an A in the course, then you know that your score on the final is at 
least 90%. If you do not receive an A, you may or may not have scored at least 90% on the 
final. Be careful not to use “q only if p” to express p > q because this is incorrect. The word 
“only” plays an essential role here. To see this, note that the truth values of “q only if p” and 
p > q are different when p and q have different truth values. To see why “q is necessary for p” 
is equivalent to “if p, then q,” observe that “q is necessary for p” means that p cannot be true 
unless q is true, or that if q is false, then p is false. This is the same as saying that if p is true, 
then q is true. To see why “p is sufficient for q” is equivalent to “if p, then q,” note that “p is 
sufficient for q?” means if p is true, it must be the case that q is also true. This is the same as 
saying that if p is true, then q is also true. 

To remember that “q unless sp” expresses the same conditional statement as “if p, then 
q,” note that “q unless =p” means that if =p is false, then q must be true. That is, the state- 
ment “q unless =p” is false when p is true but q is false, but it is true otherwise. Consequently, 
“q unless =p” and p > q always have the same truth value. 
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We illustrate the translation between conditional statements and English statements in 
Example 10. 


Let p be the statement “Maria learns discrete mathematics” and q the statement “Maria will find 
a good job.” Express the statement p > q as a statement in English. 


Solution: From the definition of conditional statements, we see that when p is the statement 
“Maria learns discrete mathematics” and q is the statement “Maria will find a good job,” p > q 
represents the statement 


“If Maria learns discrete mathematics, then she will find a good job.” 


There are many other ways to express this conditional statement in English. Among the most 
natural of these are 


“Maria will find a good job when she learns discrete mathematics.” 
“For Maria to get a good job, it is sufficient for her to learn discrete mathematics.” 
and 


“Maria will find a good job unless she does not learn discrete mathematics.” < 


Note that the way we have defined conditional statements is more general than the meaning 
attached to such statements in the English language. For instance, the conditional statement in 
Example 10 and the statement 


“Tf it is sunny, then we will go to the beach” 


are statements used in normal language where there is a relationship between the hypothesis 
and the conclusion. Further, the first of these statements is true unless Maria learns discrete 
mathematics, but she does not get a good job, and the second is true unless it is indeed sunny, 
but we do not go to the beach. On the other hand, the statement 


“If Juan has a smartphone, then 2 + 3 = 5” 


is true from the definition of a conditional statement, because its conclusion is true. (The truth 
value of the hypothesis does not matter then.) The conditional statement 


“If Juan has a smartphone, then 2 + 3 = 6” 


is true if Juan does not have a smartphone, even though 2 + 3 = 6 is false. We would not use 
these last two conditional statements in natural language (except perhaps in sarcasm), because 
there is no relationship between the hypothesis and the conclusion in either statement. In math- 
ematical reasoning, we consider conditional statements of a more general sort than we use in 
English. The mathematical concept of a conditional statement is independent of a cause-and- 
effect relationship between hypothesis and conclusion. Our definition of a conditional statement 
specifies its truth values; it is not based on English usage. Propositional language is an artificial 
language; we only parallel English usage to make it easy to use and remember. 

The if-then construction used in many programming languages is different from that 
used in logic. Most programming languages contain statements such as if p then S, where 
p is a proposition and S$ is a program segment (one or more statements to be executed). 
(Although this looks as if it might be a conditional statement, S is not a proposition, but 
rather is a set of executable instructions.) When execution of a program encounters such 
a statement, S is executed if p is true, but S is not executed if p is false, as illustrated in 
Example 11. 


EXAMPLE 11 


Remember that the 
contrapositive, but 
neither the converse or 
inverse, of a conditional 
statement is equivalent 
to it. 


EXAMPLE 12 
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What is the value of the variable x after the statement 
if 2 +2 = 4 then x:=x+ 1 


if x = 0 before this statement is encountered? (The symbol := stands for assignment. The state- 
ment x := x + 1 means the assignment of the value of x + 1 to x.) 


Solution: Because 2 + 2 = 4 is true, the assignment statement x := x + 1 is executed. Hence, x 
has the value 0 + 1 = | after this statement is encountered. < 


CONVERSE, CONTRAPOSITIVE, AND INVERSE We can form some new conditional 
statements starting with a conditional statement p — q. In particular, there are three related 
conditional statements that occur so often that they have special names. The proposition q > p 
is called the converse of p > q. The contrapositive of p > q is the proposition =q —> ~p. The 
proposition =p > ~q is called the inverse of p > q. We will see that of these three condi- 
tional statements formed from p > q, only the contrapositive always has the same truth value 
as p >q. 

We first show that the contrapositive, =q > =p, of a conditional statement p > q always 
has the same truth value as p > q. To see this, note that the contrapositive is false only when =p 
is false and ~q is true, that is, only when p is true and q is false. We now show that neither the 
converse, q > p, nor the inverse, =p > ~q, has the same truth value as p > q for all possible 
truth values of p and q. Note that when p is true and q is false, the original conditional statement 
is false, but the converse and the inverse are both true. 

When two compound propositions always have the same truth values, regardless of the truth 
values of its propositional variables, we call them equivalent. Hence, a conditional statement 
and its contrapositive are equivalent. The converse and the inverse of a conditional statement 
are also equivalent, as the reader can verify, but neither is equivalent to the original conditional 
statement. (We will study equivalent propositions in Section 1.3.) Take note that one of the most 
common logical errors is to assume that the converse or the inverse of a conditional statement 
is equivalent to this conditional statement. 

We illustrate the use of conditional statements in Example 12. 


Find the contrapositive, the converse, and the inverse of the conditional statement 


“The home team wins whenever it is raining.” 


Solution: Because “q whenever p” is one of the ways to express the conditional statement p > q, 
the original statement can be rewritten as 


“Tf it is raining, then the home team wins.” 
Consequently, the contrapositive of this conditional statement is 
“If the home team does not win, then it is not raining.” 
The converse is 
“If the home team wins, then it is raining.” 
The inverse is 
“Tf it is not raining, then the home team does not win.” 


Only the contrapositive is equivalent to the original statement. < 
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BICONDITIONALS We now introduce another way to combine propositions that expresses 
that two propositions have the same truth value. 


Let p and q be propositions. The biconditional statement p © q is the proposition “p if and 
only if q.” The biconditional statement p < q is true when p and q have the same truth values, 
and is false otherwise. Biconditional statements are also called bi-implications. 


The truth table for p + qis shown in Table 6. Note that the statement p <+ q is true when both the 
conditional statements p > q and q > p are true and is false otherwise. That is why we use the 
words “if and only if” to express this logical connective and why it is symbolically written by 
combining the symbols —> and <. There are some other common ways to express p © q: 


“p is necessary and sufficient for q” 

“if p then q, and conversely” 

“p iff q.” “p exactly when q.” 
The last way of expressing the biconditional statement p + q uses the abbreviation “iff” for “if 
and only if.” Note that p + q has exactly the same truth value as (p > q) A (q > p). 


Let p be the statement “You can take the flight,” and let q be the statement “You buy a ticket.” 
Then p © q is the statement 


“You can take the flight if and only if you buy a ticket.” 


This statement is true if p and q are either both true or both false, that is, if you buy a ticket and 
can take the flight or if you do not buy a ticket and you cannot take the flight. It is false when 
p and q have opposite truth values, that is, when you do not buy a ticket, but you can take the 
flight (such as when you get a free trip) and when you buy a ticket but you cannot take the flight 
(such as when the airline bumps you). < 


IMPLICIT USE OF BICONDITIONALS You should be aware that biconditionals are not 
always explicit in natural language. In particular, the “if and only if’ construction used in 
biconditionals is rarely used in common language. Instead, biconditionals are often expressed 
using an “if, then” or an “only if” construction. The other part of the “if and only if” is implicit. 
That is, the converse is implied, but not stated. For example, consider the statement in English 
“If you finish your meal, then you can have dessert.” What is really meant is “You can have 
dessert if and only if you finish your meal.” This last statement is logically equivalent to the 
two statements “If you finish your meal, then you can have dessert” and “You can have dessert 
only if you finish your meal.” Because of this imprecision in natural language, we need to 
make an assumption whether a conditional statement in natural language implicitly includes its 
converse. Because precision is essential in mathematics and in logic, we will always distinguish 
between the conditional statement p > q and the biconditional statement p © q. 
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1.1.4 Truth Tables of Compound Propositions 


We have now introduced five important logical connectives—conjunction, disjunction, exclu- 
sive or, implication, and the biconditional operator—as well as negation. We can use these con- 
nectives to build up complicated compound propositions involving any number of propositional 
variables. We can use truth tables to determine the truth values of these compound propositions, 
as Example 14 illustrates. We use a separate column to find the truth value of each compound 
expression that occurs in the compound proposition as it is built up. The truth values of the 
compound proposition for each combination of truth values of the propositional variables in it 
is found in the final column of the table. 


Construct the truth table of the compound proposition 


(pV 7q) > (PAQq). 


Solution: Because this truth table involves two propositional variables p and q, there are four 
rows in this truth table, one for each of the pairs of truth values TT, TF, FT, and FF. The first two 
columns are used for the truth values of p and q, respectively. In the third column we find the 
truth value of ~q, needed to find the truth value of p V ~q, found in the fourth column. The fifth 
column gives the truth value of p A q. Finally, the truth value of (p V =q) > (p A q) is found in 
the last column. The resulting truth table is shown in Table 7. < 


TABLE 7 The Truth Table of (p v ~q) > (p ^q). 


ETENEE ev eas) 
T 
T 
F 
T 


F 
F 
F 


1.1.5 Precedence of Logical Operators 


We can construct compound propositions using the negation operator and the logical operators 
defined so far. We will generally use parentheses to specify the order in which logical operators 
in a compound proposition are to be applied. For instance, (p V q) A (=r) is the conjunction 
of p V q and ~r. However, to reduce the number of parentheses, we specify that the negation 
operator is applied before all other logical operators. This means that =p A q is the conjunction 
of ap and q, namely, (=p) A q, not the negation of the conjunction of p and q, namely =(p A q). (It 
is generally the case that unary operators that involve only one object precede binary operators.) 

Another general rule of precedence is that the conjunction operator takes precedence over 
the disjunction operator, so that p V q Ar means p V (q Ar) rather than (p V gq) ArandpAqvr 
means (p A q) V r rather than p A (q V r). Because this rule may be difficult to remember, we will 
continue to use parentheses so that the order of the disjunction and conjunction operators is clear. 

Finally, it is an accepted rule that the conditional and biconditional operators, > and «, 
have lower precedence than the conjunction and disjunction operators, A and V. Consequently, 
p > qV r means p > (q V r) rather than (p > q) Vr and p V q > r means (p V q) > r rather 
than p V (q > r). We will use parentheses when the order of the conditional operator and bi- 
conditional operator is at issue, although the conditional operator has precedence over the 
biconditional operator. Table 8 displays the precedence levels of the logical operators, =, A, 
V, >, and o. 
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1.1.6 Logic and Bit Operations 


Computers represent information using bits. A bit is a symbol with two possible values, namely, 
0 (zero) and 1 (one). This meaning of the word bit comes from binary digit, because zeros and 
ones are the digits used in binary representations of numbers. The well-known statistician John 
Tukey introduced this terminology in 1946. A bit can be used to represent a truth value, because 
there are two truth values, namely, true and false. As is customarily done, we will use a 1 bit 
to represent true and a 0 bit to represent false. That is, 1 represents T (true), O represents F 
(false). A variable is called a Boolean variable if its value is either true or false. Consequently, 
a Boolean variable can be represented using a bit. 

Computer bit operations correspond to the logical connectives. By replacing true by a one 
and false by a zero in the truth tables for the operators A, V, and @, the columns in Table 9 for 
the corresponding bit operations are obtained. We will also use the notation OR, AND, and XOR 
for the operators v, A, and @, as is done in various programming languages. 


TABLE 9 Table for the Bit Operators OR, 
AND, and XOR. 


æ 
0 


= OC © 


Information is often represented using bit strings, which are lists of zeros and ones. When 
this is done, operations on the bit strings can be used to manipulate this information. 


A bit string is a sequence of zero or more bits. The length of this string is the number of bits 
in the string. 


101010011 is a bit string of length nine. < 


JOHN WILDER TUKEY (1915-2000) Tukey, born in New Bedford, Massachusetts, was an only child. 
His parents, both teachers, decided home schooling would best develop his potential. His formal education 
began at Brown University, where he studied mathematics and chemistry. He received a master’s degree in 
chemistry from Brown and continued his studies at Princeton University, changing his field of study from 
chemistry to mathematics. He received his Ph.D. from Princeton in 1939 for work in topology, when he 
was appointed an instructor in mathematics at Princeton. With the start of World War II, he joined the Fire 
Control Research Office, where he began working in statistics. Tukey found statistical research to his liking 
and impressed several leading statisticians with his skills. In 1945, at the conclusion of the war, Tukey re- 
turned to the mathematics department at Princeton as a professor of statistics, and he also took a position 
at AT&T Bell Laboratories. Tukey founded the Statistics Department at Princeton in 1966 and was its first 
chairman. Tukey made significant contributions to many areas of statistics, including the analysis of variance, 


the estimation of spectra of time series, inferences about the values of a set of parameters from a single experiment, and the philos- 
ophy of statistics. However, he is best known for his invention, with J. W. Cooley, of the fast Fourier transform. In addition to his 
contributions to statistics, Tukey was noted as a skilled wordsmith; he is credited with coining the terms bit and software. 

Tukey contributed his insight and expertise by serving on the President’s Science Advisory Committee. He chaired several 
important committees dealing with the environment, education, and chemicals and health. He also served on committees working 
on nuclear disarmament. Tukey received many awards, including the National Medal of Science. 


HISTORICAL NOTE There were several other suggested words for a binary digit, including binit and bigit, that never were widely 
accepted. The adoption of the word bit may be due to its meaning as acommon English word. For an account of Tukey’s coining of 
the word bit, see the April 1984 issue of Annals of the History of Computing. 
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We can extend bit operations to bit strings. We define the bitwise OR, bitwise AND, and 
bitwise XOR of two strings of the same length to be the strings that have as their bits the OR, 
AND, and XOR of the corresponding bits in the two strings, respectively. We use the symbols 
V, A, and È to represent the bitwise OR, bitwise AND, and bitwise XOR operations, respectively. 
We illustrate bitwise operations on bit strings with Example 16. 


EXAMPLE 16 Find the bitwise OR, bitwise AND, and bitwise XOR of the bit strings 0110110110 and 
110001 1101. (Here, and throughout this book, bit strings will be split into blocks of four bits 


to make them easier to read.) 


Solution: The bitwise OR, bitwise AND, and bitwise XOR of these strings are obtained by taking 
the OR, AND, and XOR of the corresponding bits, respectively. This gives us 


01 1011 0110 
11 0001 1101 


11 1011 1111 bitwise OR 
01 0001 0100 bitwise AND 
10 1010 1011 bitwise XOR 


Exercises 


. Which of these sentences are propositions? What are the 
truth values of those that are propositions? 
a) Boston is the capital of Massachusetts. 
b) Miami is the capital of Florida. 


c) 24+3=5. 
d) 5+7= 10. 
e) x+2=11. 


f) Answer this question. 


. Which of these are propositions? What are the truth 
values of those that are propositions? 


a) Do not pass go. 
b) What time is it? 
c) There are no black flies in Maine. 
d) 4+x=5. 
e) The moon is made of green cheese. 
f) 2” > 100. 
. What is the negation of each of these propositions? 
a) Linda is younger than Sanjay. 
b) Mei makes more money than Isabella. 
c) Moshe is taller than Monica. 
d) Abby is richer than Ricardo. 
. What is the negation of each of these propositions? 
a) Janice has more Facebook friends than Juan. 
b) Quincy is smarter than Venkat. 
c) Zelda drives more miles to school than Paola. 
d) Briana sleeps longer than Gloria. 
. What is the negation of each of these propositions? 
a) Mei has an MP3 player. 
b) There is no pollution in New Jersey. 
c) 24+1=3. 
d) The summer in Maine is hot and sunny. 
. What is the negation of each of these propositions? 
a) Jennifer and Teja are friends. 
b) There are 13 items in a baker’s dozen. 


c) Abby sent more than 100 text messages yesterday. 
d) 121 is a perfect square. 


7. What is the negation of each of these propositions? 


a) Steve has more than 100 GB free disk space on his 
laptop. 

b) Zach blocks e-mails and texts from Jennifer. 

c) 7-11-13 = 999. 

d) Diane rode her bicycle 100 miles on Sunday. 


. Suppose that Smartphone A has 256 MB RAM and 


32 GB ROM, and the resolution of its camera is 8 MP; 
Smartphone B has 288 MB RAM and 64 GB ROM, and 
the resolution of its camera is 4 MP; and Smartphone C 
has 128 MB RAM and 32 GB ROM, and the resolution 
of its camera is 5 MP. Determine the truth value of each 
of these propositions. 


a) Smartphone B has the most RAM of these three 
smartphones. 

b) Smartphone C has more ROM or a higher resolution 
camera than Smartphone B. 

c) Smartphone B has more RAM, more ROM, and a 
higher resolution camera than Smartphone A. 

d) If Smartphone B has more RAM and more ROM than 
Smartphone C, then it also has a higher resolution 
camera. 

e) Smartphone A has more RAM than Smartphone B if 
and only if Smartphone B has more RAM than Smart- 
phone A. 


. Suppose that during the most recent fiscal year, the an- 


nual revenue of Acme Computer was 138 billion dollars 
and its net profit was 8 billion dollars, the annual revenue 
of Nadir Software was 87 billion dollars and its net profit 
was 5 billion dollars, and the annual revenue of Quixote 
Media was 111 billion dollars and its net profit was 
13 billion dollars. Determine the truth value of each of 
these propositions for the most recent fiscal year. 
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a) Quixote Media had the largest annual revenue. 

b) Nadir Software had the lowest net profit and Acme 
Computer had the largest annual revenue. 

c) Acme Computer had the largest net profit or Quixote 
Media had the largest net profit. 

d) If Quixote Media had the smallest net profit, then 
Acme Computer had the largest annual revenue. 

e) Nadir Software had the smallest net profit if and only 
if Acme Computer had the largest annual revenue. 

Let p and q be the propositions 


p: I bought a lottery ticket this week. 

q: I won the million dollar jackpot. 
Express each of these propositions as an English sen- 
tence. 


a) =p b) pv q ©) p>q 
d) p^q e) peq f) p > ~q 
g) p ^q h) pV (pA q) 


Let p and q be the propositions “Swimming at the New 
Jersey shore is allowed” and “Sharks have been spotted 
near the shore,” respectively. Express each of these com- 
pound propositions as an English sentence. 


a) 79 b) pAq c) =pvq 
d) p> =q e) ~q >p f) p > ~q 
g) pong h) ap A (pV 7q) 


Let p and q be the propositions “The election is decided” 
and “The votes have been counted,” respectively. Express 
each of these compound propositions as an English sen- 
tence. 


a) =p b) pvq 

c) paq d) q>p 

e) =q > =p f) =p > ~q 

g peq h) aq V p^ q) 


Let p and q be the propositions 
p: It is below freezing. 
q: It is snowing. 
Write these propositions using p and q and logical con- 
nectives (including negations). 
a) Itis below freezing and snowing. 
b) Itis below freezing but not snowing. 
c) Itis not below freezing and it is not snowing. 
d) Itis either snowing or below freezing (or both). 
e) If itis below freezing, it is also snowing. 
f) Either it is below freezing or it is snowing, but it is 
not snowing if it is below freezing. 
g) That it is below freezing is necessary and sufficient 
for it to be snowing. 
Let p, q, and r be the propositions 
p: You have the flu. 
q: You miss the final examination. 
r: You pass the course. 
Express each of these propositions as an English sen- 
tence. 
a) p>q 
c) q>r 
e) (p > =r) V (q > `r) 


b) aq er 
d) pVqvr 
f) paq) Yq r) 
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Let p and q be the propositions 


p: You drive over 65 miles per hour. 
q: You get a speeding ticket. 


Write these propositions using p and q and logical con- 

nectives (including negations). 

a) You do not drive over 65 miles per hour. 

b) You drive over 65 miles per hour, but you do not get 
a speeding ticket. 

c) You will get a speeding ticket if you drive over 
65 miles per hour. 

d) If you do not drive over 65 miles per hour, then you 
will not get a speeding ticket. 

e) Driving over 65 miles per hour is sufficient for getting 
a speeding ticket. 

f) You get a speeding ticket, but you do not drive over 
65 miles per hour. 

g) Whenever you get a speeding ticket, you are driving 
over 65 miles per hour. 


Let p, g, and r be the propositions 


p: You get an A on the final exam. 
q: You do every exercise in this book. 
r: You get an A in this class. 


Write these propositions using p, q, and r and logical con- 

nectives (including negations). 

a) You get an A in this class, but you do not do every 
exercise in this book. 

b) You get an A on the final, you do every exercise in 
this book, and you get an A in this class. 

c) To get an A in this class, it is necessary for you to get 
an A on the final. 

d) You get an A on the final, but you don’t do every ex- 
ercise in this book; nevertheless, you get an A in this 
class. 

e) Getting an A on the final and doing every exercise in 
this book is sufficient for getting an A in this class. 

f) You will get an A in this class if and only if you either 
do every exercise in this book or you get an A on the 
final. 


Let p, q, and r be the propositions 


p: Grizzly bears have been seen in the area. 
q: Hiking is safe on the trail. 
r: Berries are ripe along the trail. 


Write these propositions using p, q, and r and logical con- 

nectives (including negations). 

a) Berries are ripe along the trail, but grizzly bears have 
not been seen in the area. 

b) Grizzly bears have not been seen in the area and hik- 
ing on the trail is safe, but berries are ripe along the 
trail. 

c) If berries are ripe along the trail, hiking is safe if and 
only if grizzly bears have not been seen in the area. 

d) Itis not safe to hike on the trail, but grizzly bears have 
not been seen in the area and the berries along the trail 
are ripe. 

e) For hiking on the trail to be safe, it is necessary but 
not sufficient that berries not be ripe along the trail 
and for grizzly bears not to have been seen in the area. 
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f) Hiking is not safe on the trail whenever grizzly bears 
have been seen in the area and berries are ripe along 
the trail. 

Determine whether these biconditionals are true or 

false. 

a) 24+2=4ifand only if 1 +1 = 2. 

b) 1+1 = 2 if and only if 2 +3 = 4. 

c) 1+ 1 = 3 if and only if monkeys can fly. 

d) 0 > 1 if and only if 2 > 1. 

Determine whether each of these conditional statements 

is true or false. 

a) If 1+1 = 2, then 2+2 = 5. 

b) If 1 + 1 = 3, then 2 +2 = 4. 

c) If1 +1 = 3, then2+2=5. 

d) If monkeys can fly, then 1 + 1 = 3. 

Determine whether each of these conditional statements 

is true or false. 

a) If 1+ 1 = 3, then unicorns exist. 

b) If 1 + 1 = 3, then dogs can fly. 

c) If 1+1 = 2, then dogs can fly. 

d) If2 +2 = 4, then 1 +2 =3. 

For each of these sentences, determine whether an in- 

clusive or, or an exclusive or, is intended. Explain your 

answer. 

a) Coffee or tea comes with dinner. 

b) A password must have at least three digits or be at 
least eight characters long. 

c) The prerequisite for the course is a course in number 
theory or a course in cryptography. 

d) You can pay using U.S. dollars or euros. 

For each of these sentences, determine whether an in- 

clusive or, or an exclusive or, is intended. Explain your 

answer. 

a) Experience with C++ or Java is required. 

b) Lunch includes soup or salad. 

c) To enter the country you need a passport or a voter 
registration card. 

d) Publish or perish. 

For each of these sentences, state what the sentence 

means if the logical connective or is an inclusive or (that 

is, a disjunction) versus an exclusive or. Which of these 
meanings of or do you think is intended? 

a) To take discrete mathematics, you must have taken 
calculus or a course in computer science. 

b) When you buy a new car from Acme Motor Company, 
you get $2000 back in cash or a 2% car loan. 

c) Dinner for two includes two items from column A or 
three items from column B. 

d) School is closed if more than two feet of snow falls or 
if the wind chill is below —100 °F. 

Write each of these statements in the form “if p, then q” 

in English. [Hint: Refer to the list of common ways to ex- 

press conditional statements provided in this section. ] 

a) It is necessary to wash the boss’s car to get promoted. 

b) Winds from the south imply a spring thaw. 


25. 
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27. 
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c) A sufficient condition for the warranty to be good is 
that you bought the computer less than a year ago. 

d) Willy gets caught whenever he cheats. 

e) You can access the website only if you pay a subscrip- 
tion fee. 

f) Getting elected follows from knowing the right 
people. 

g) Carol gets seasick whenever she is on a boat. 

Write each of these statements in the form “if p, then q” 

in English. [Hint: Refer to the list of common ways to 

express conditional statements. ] 


a) It snows whenever the wind blows from the northeast. 

b) The apple trees will bloom if it stays warm for 
a week. 

c) That the Pistons win the championship implies that 
they beat the Lakers. 

d) It is necessary to walk eight miles to get to the top of 
Long’s Peak. 

e) To get tenure as a professor, it is sufficient to be world 
famous. 

f) If you drive more than 400 miles, you will need to 
buy gasoline. 

g) Your guarantee is good only if you bought your CD 
player less than 90 days ago. 

h) Jan will go swimming unless the water is too cold. 

i) We will have a future, provided that people believe in 
science. 


Write each of these statements in the form “if p, then q” 
in English. [Hint: Refer to the list of common ways to ex- 
press conditional statements provided in this section. ] 


a) I will remember to send you the address only if you 
send me an e-mail message. 

b) To bea citizen of this country, it is sufficient that you 
were born in the United States. 

c) Ifyou keep your textbook, it will be a useful reference 
in your future courses. 

d) The Red Wings will win the Stanley Cup if their 
goalie plays well. 

e) That you get the job implies that you had the best cre- 
dentials. 

f) The beach erodes whenever there is a storm. 

g) It is necessary to have a valid password to log on to 
the server. 

h) You will reach the summit unless you begin your 
climb too late. 

i) You will get a free ice cream cone, provided that you 
are among the first 100 customers tomorrow. 


Write each of these propositions in the form “p if and 

only if q” in English. 

a) If it is hot outside you buy an ice cream cone, and if 
you buy an ice cream cone it is hot outside. 

b) For you to win the contest it is necessary and suffi- 
cient that you have the only winning ticket. 

c) You get promoted only if you have connections, and 
you have connections only if you get promoted. 

d) Ifyou watch television your mind will decay, and con- 
versely. 

e) The trains run late on exactly those days when I 
take it. 


16 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


1 / The Foundations: Logic and Proofs 


Write each of these propositions in the form “p if and 

only if q” in English. 

a) For you to get an A in this course, it is necessary and 
sufficient that you learn how to solve discrete mathe- 
matics problems. 

b) If you read the newspaper every day, you will be in- 
formed, and conversely. 

c) Itrains if it is a weekend day, and it is a weekend day 
if it rains. 

d) You can see the wizard only if the wizard is not in, 
and the wizard is not in only if you can see him. 

e) My airplane flight is late exactly when I have to catch 
a connecting flight. 

State the converse, contrapositive, and inverse of each of 

these conditional statements. 

a) If it snows today, I will ski tomorrow. 

b) Icome to class whenever there is going to be a quiz. 

c) A positive integer is a prime only if it has no divisors 
other than 1 and itself. 

State the converse, contrapositive, and inverse of each of 

these conditional statements. 

a) If it snows tonight, then I will stay at home. 

b) I go to the beach whenever it is a sunny summer day. 

c) When I stay up late, it is necessary that I sleep until 
noon. 

How many rows appear in a truth table for each of these 

compound propositions? 

a) p> 7p 

b) PV =r) A (GV >s) 

co) qVpV7SV>rvVv-atvu 

d) (parade (GAD 

How many rows appear in a truth table for each of these 

compound propositions? 

a) (4 > =p) V Cp > ~q) 

b) (pV 7H) ^A (p V 7s) 

c) (p >r) vV Gs > 7t) V Cu > v) 

d) (PATAS)V(GANV(TA 7H) 

Construct a truth table for each of these compound propo- 

sitions. 


a) pA7p 

© PY") > 4 

e) p > q9) 2 Cq > p) 
f) p >49) > (>p) 
Construct a truth table for each of these compound propo- 
sitions. 


a) p> 7p 

c) POV) 

e) (q > m) p 2q) 
f) p4) P © q) 
Construct a truth table for each of these compound propo- 
sitions. 


a) py) > (Gq) 
© pya) 9 Pp^q) 
e) p 24)9 Op © `r) 
f) (p®q) > pq) 


b) pV 7p 
d) PV) > pP^q) 


b) p e =p 
d) p^q)> PVq) 


b) P®g) > PAQ 
d pege(pe q) 
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Construct a truth table for each of these compound propo- 
sitions. 


a) p®p b) p®-p 

c) pO-g d) ~p ®-q 

e) P ®q)V P99) f) pqa) ^p q) 
Construct a truth table for each of these compound propo- 
sitions. 

a) p> =q 

9 p>4)V p> 9 
e) p2q4)V Cp 2q) 
f) Œp ° =q) P © q) 
Construct a truth table for each of these compound propo- 
sitions. 


b) pq 
d) p >q) Cp >q) 


a) (pvq)vr b) (PY) Ar 
c) (p^q) vr d) (PAQAr 
e) (PY) Ar f) p^q) Vr 


Construct a truth table for each of these compound propo- 
sitions. 

a) p> qvr) 

b) p> (q >r) 

©) p>q4)Vp>r) 

d) p >4)^Cp >r) 

e) (peq) Cq © r) 

f) p e q) ger) 

Construct a truth table for ((p > q) > r) > s. 
Construct a truth table for (p e q) e (r e 5). 

Explain, without using a truth table, why (p V =q) A 
(q Var) A (r V =p) is true when p, q, and r have the same 
truth value and it is false otherwise. 

Explain, without using a truth table, why (pV qV r)^A 
(p V ~q V `r) is true when at least one of p, q, and r is 
true and at least one is false, but is false when all three 
variables have the same truth value. 

If Pi, Po ---, P, are n propositions, explain why 


n-l n 


VAN VAN (=p; V =p;) 


i=l j=i+1 


is true if and only if at most one of p4, Pz, ---, Pn 18 true. 

Use Exercise 44 to construct a compound proposition 
that is true if and only if exactly one of the proposi- 
tions pı, P2, ..., Pn is true. [Hint: Combine the compound 
proposition in Exercise 44 and a compound proposition 
that is true if and only if at least one of pj, py, ..., Pn 18 
true. | 

What is the value of x after each of these statements is 
encountered in a computer program, if x = 1 before the 
statement is reached? 

a) ifx+2 = 3 then x:=x+ 1 

b) if (x + 1 = 3) OR (2x + 2 = 3) then x :=x + 1 

c) if (2x +3 = 5) AND (3x + 4 = 7) then x := x+ 1 

d) if œ+ 1 = 2) XOR (x + 2 = 3) then x := x + 1 

e) ifx< 2 then x := x+ 1 

Find the bitwise OR, bitwise AND, and bitwise XOR of 
each of these pairs of bit strings. 

a) 101 1110, 010 0001 

b) 1111 0000, 1010 1010 

c) 000111 0001, 100100 1000 

d) 11 1111 1111, 00 0000 0000 


48. Evaluate each of these expressions. 
a) 11000A (0 1011 v 1 1011) 
b) (0 1111 A 10101) v 0 1000 
c) (0 1010 @ 1 1011) ® O 1000 
d) (1 1011 v 0 1010) A (1 0001 v 1 1011) 

Fuzzy logic is used in artificial intelligence. In fuzzy logic, a 

proposition has a truth value that is a number between 0 and 

1, inclusive. A proposition with a truth value of 0 is false and 

one with a truth value of 1 is true. Truth values that are be- 

tween 0 and 1 indicate varying degrees of truth. For instance, 
the truth value 0.8 can be assigned to the statement “Fred is 
happy,” because Fred is happy most of the time, and the truth 
value 0.4 can be assigned to the statement “John is happy,” 
because John is happy slightly less than half the time. Use 

these truth values to solve Exercises 49-51. 

49. The truth value of the negation of a proposition in fuzzy 
logic is 1 minus the truth value of the proposition. What 
are the truth values of the statements “Fred is not happy” 
and “John is not happy”? 

50. The truth value of the conjunction of two propositions in 
fuzzy logic is the minimum of the truth values of the two 
propositions. What are the truth values of the statements 


51. 
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“Fred and John are happy” and “Neither Fred nor John is 
happy”? 

The truth value of the disjunction of two propositions in 
fuzzy logic is the maximum of the truth values of the two 
propositions. What are the truth values of the statements 
“Fred is happy, or John is happy” and “Fred is not happy, 
or John is not happy”? 


Is the assertion “This statement is false” a proposition? 


The nth statement in a list of 100 statements is “Exactly 

n of the statements in this list are false.” 

a) What conclusions can you draw from these state- 
ments? 

b) Answer part (a) if the nth statement is “At least n of 
the statements in this list are false.” 

c) Answer part (b) assuming that the list contains 99 
statements. 

An ancient Sicilian legend says that the barber in a remote 

town who can be reached only by traveling a dangerous 

mountain road shaves those people, and only those peo- 

ple, who do not shave themselves. Can there be such a 

barber? 


Applications of Propositional Logic 


1.2.1 Introduction 


Logic has many important applications to mathematics, computer science, and numerous other 
disciplines. Statements in mathematics and the sciences and in natural language often are im- 
precise or ambiguous. To make such statements precise, they can be translated into the language 
of logic. For example, logic is used in the specification of software and hardware, because these 
specifications need to be precise before development begins. Furthermore, propositional logic 
and its rules can be used to design computer circuits, to construct computer programs, to ver- 
ify the correctness of programs, and to build expert systems. Logic can be used to analyze and 
solve many familiar puzzles. Software systems based on the rules of logic have been developed 
for constructing some, but not all, types of proofs automatically. We will discuss some of these 
applications of propositional logic in this section and in later chapters. 


1.2.2 Translating English Sentences 


There are many reasons to translate English sentences into expressions involving propositional 
variables and logical connectives. In particular, English (and every other human language) is 
often ambiguous. Translating sentences into compound statements (and other types of logical 
expressions, which we will introduce later in this chapter) removes the ambiguity. Note that 
this may involve making a set of reasonable assumptions based on the intended meaning of 
the sentence. Moreover, once we have translated sentences from English into logical expres- 
sions, we can analyze these logical expressions to determine their truth values, we can manip- 
ulate them, and we can use rules of inference (which are discussed in Section 1.6) to reason 


about them. 


To illustrate the process of translating an English sentence into a logical expression, con- 


sider Examples 1 and 2. 
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EXAMPLE 1 


Extra 
Examples 


» 


EXAMPLE 2 


EXAMPLE 3 


Extra 
Examples 


> 


How can this English sentence be translated into a logical expression? 


“You can access the Internet from campus only if you are a computer science major or you 
are not a freshman.” 


Solution: There are many ways to translate this sentence into a logical expression. Although it is 
possible to represent the sentence by a single propositional variable, such as p, this would not be 
useful when analyzing its meaning or reasoning with it. Instead, we will use propositional vari- 
ables to represent each sentence part and determine the appropriate logical connectives between 
them. In particular, we let a, c, and f represent “You can access the Internet from campus,” “You 
are a computer science major,” and “You are a freshman,” respectively. Noting that “only if” is 
one way a conditional statement can be expressed, this sentence can be represented as 


a> (cv -f). < 


How can this English sentence be translated into a logical expression? 


“You cannot ride the roller coaster if you are under 4 feet tall unless you are older than 16 
years old.” 


Solution: Let q, r, and s represent “You can ride the roller coaster,” “You are under 4 feet tall,” 
and “You are older than 16 years old,” respectively. Then the sentence can be translated to 


(r Ans) > ~q. 


There are other ways to represent the original sentence as a logical expression, but the one 
we have used should meet our needs. < 


1.2.3 System Specifications 


Translating sentences in natural language (such as English) into logical expressions is an essen- 
tial part of specifying both hardware and software systems. System and software engineers take 
requirements in natural language and produce precise and unambiguous specifications that can 
be used as the basis for system development. Example 3 shows how compound propositions can 
be used in this process. 


Express the specification “The automated reply cannot be sent when the file system is full” 
using logical connectives. 


Solution: One way to translate this is to let p denote “The automated reply can be sent” and q de- 
note “The file system is full.” Then ~p represents “It is not the case that the automated reply can 
be sent,” which can also be expressed as “The automated reply cannot be sent.” Consequently, 
our specification can be represented by the conditional statement q > =p. < 


System specifications should be consistent, that is, they should not contain conflicting re- 
quirements that could be used to derive a contradiction. When specifications are not consistent, 
there would be no way to develop a system that satisfies all specifications. 


EXAMPLE 4 Determine whether these system specifications are consistent: 


“The diagnostic message is stored in the buffer or it is retransmitted.” 
“The diagnostic message is not stored in the buffer.” 
“Tf the diagnostic message is stored in the buffer, then it is retransmitted.” 
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Solution: To determine whether these specifications are consistent, we first express them using 
logical expressions. Let p denote “The diagnostic message is stored in the buffer” and let q 
denote “The diagnostic message is retransmitted.” The specifications can then be written as 
DV q, “p, and p > q. An assignment of truth values that makes all three specifications true 
must have p false to make ~p true. Because we want p V q to be true but p must be false, q must 
be true. Because p > q is true when p is false and q is true, we conclude that these specifications 
are consistent, because they are all true when p is false and q is true. We could come to the same 
conclusion by use of a truth table to examine the four possible assignments of truth values to p 
and q. < 


Do the system specifications in Example 4 remain consistent if the specification “The diagnostic 
message is not retransmitted” is added? 


Solution: By the reasoning in Example 4, the three specifications from that example are true 
only in the case when p is false and q is true. However, this new specification is =q, which is 
false when q is true. Consequently, these four specifications are inconsistent. < 


1.2.4 Boolean Searches 


Logical connectives are used extensively in searches of large collections of information, such 
as indexes of Web pages. Because these searches employ techniques from propositional logic, 
they are called Boolean searches. 

In Boolean searches, the connective AND is used to match records that contain both of 
two search terms, the connective OR is used to match one or both of two search terms, and the 
connective NOT (sometimes written as AND NOT ) is used to exclude a particular search term. 
Careful planning of how logical connectives are used is often required when Boolean searches 
are used to locate information of potential interest. Example 6 illustrates how Boolean searches 
are carried out. 


Web Page Searching Most Web search engines support Boolean searching techniques, which 
is useful for finding Web pages about particular subjects. For instance, using Boolean searching 
to find Web pages about universities in New Mexico, we can look for pages matching NEW 
AND MEXICO AND UNIVERSITIES. The results of this search will include those pages that 
contain the three words NEW, MEXICO, and UNIVERSITIES. This will include all of the 
pages of interest, together with others such as a page about new universities in Mexico. (Note 
that Google, and many other search engines, do require the use of “AND” because such search 
engines use all search terms by default.) Most search engines support the use of quotation marks 
to search for specific phrases. So, it may be more effective to search for pages matching “NEW 
MEXICO” AND UNIVERSITIES. 

Next, to find pages that deal with universities in New Mexico or Arizona, we can search 
for pages matching (NEW AND MEXICO OR ARIZONA) AND UNIVERSITIES. (Note: Here 
the AND operator takes precedence over the OR operator. Also, in Google, the terms used for 
this search would be NEW MEXICO OR ARIZONA.) The results of this search will include 
all pages that contain the word UNIVERSITIES and either both the words NEW and MEXICO 
or the word ARIZONA. Again, pages besides those of interest will be listed. Finally, to find 
Web pages that deal with universities in Mexico (and not New Mexico), we might first look 
for pages matching MEXICO AND UNIVERSITIES, but because the results of this search will 
include pages about universities in New Mexico, as well as universities in Mexico, it might be 
better to search for pages matching (MEXICO AND UNIVERSITIES) NOT NEW. The results 
of this search include pages that contain both the words MEXICO and UNIVERSITIES but 
do not contain the word NEW. (In Google, and many other search engines, the word “NOT” is 
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replaced by the symbol “-”. In Google, the terms used for this last search would be MEXICO 
UNIVERSITIES -NEW.) < 


1.2.5 Logic Puzzles 


Puzzles that can be solved using logical reasoning are known as logic puzzles. Solving logic 
puzzles is an excellent way to practice working with the rules of logic. Also, computer programs 
designed to carry out logical reasoning often use well-known logic puzzles to illustrate their 
capabilities. Many people enjoy solving logic puzzles, published in periodicals, books, and on 
the Web, as a recreational activity. 

The next three examples present logic puzzles, in increasing level of difficulty. Many others 
can be found in the exercises. In Section 1.3 we will discuss the n-queens puzzle and the game 
of Sudoku. 


As a reward for saving his daughter from pirates, the King has given you the opportunity to win 
a treasure hidden inside one of three trunks. The two trunks that do not hold the treasure are 
empty. To win, you must select the correct trunk. Trunks 1 and 2 are each inscribed with the 
message “This trunk is empty,” and Trunk 3 is inscribed with the message “The treasure is in 
Trunk 2.” The Queen, who never lies, tells you that only one of these inscriptions is true, while 
the other two are wrong. Which trunk should you select to win? 


Solution: Let p; be the proposition that the treasure is in Trunk i, for i = 1, 2, 3. To translate into 
propositional logic the Queen’s statement that exactly one of the inscriptions is true, we observe 
that the inscriptions on Trunk 1, Trunk 2, and Trunk 3, are =p,, ap,, and p}, respectively. So, 
her statement can be translated to 


(=p, A (7p 2) A mPa) V (ACP) A P2 A P2) V OOP) A AP 2) A P2))- 


Using the rules for propositional logic, we see that this is equivalent to (p; A =p) V (p; A p2). By 
the distributive law, (p; A =p2) V (p1 A p2) is equivalent to p; A (=p, V pz). But because =p, V 
P must be true, this is then equivalent to p} A T, which is in turn equivalent to p,. So the treasure 
is in Trunk 1 (that is, p, is true), and p, and p, are false; and the inscription on Trunk 2 is the 
only true one. (Here, we have used the concept of propositional equivalence, which is discussed 
in Section 1.3.) < 


Next, we introduce a puzzle originally posed by Raymond Smullyan, a master of logic 
puzzles, who has published more than a dozen books containing challenging puzzles that involve 
logical reasoning. 


In [Sm78] Smullyan posed many puzzles about an island that has two kinds of inhabitants, 
knights, who always tell the truth, and their opposites, knaves, who always lie. You encounter 
two people A and B. What are A and B if A says “B is a knight’ and B says “The two of us are 
opposite types”? 


Solution: Let p and q be the statements that A is a knight and B is a knight, respectively, so that 
ap and ~q are the statements that A is a knave and B is a knave, respectively. 

We first consider the possibility that A is a knight; this is the statement that p is true. If A is 
a knight, then he is telling the truth when he says that B is a knight, so that g is true, and A and B 
are the same type. However, if B is a knight, then B’s statement that A and B are of opposite 
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types, the statement (p A ~q) V (=p ^ q), would have to be true, which it is not, because A and 
B are both knights. Consequently, we can conclude that A is not a knight, that is, that p is false. 

If A is a knave, then because everything a knave says is false, A’s statement that B is 
a knight, that is, that q is true, is a lie. This means that q is false and B is also a knave. 
Furthermore, if B is a knave, then B’s statement that A and B are opposite types is a lie, 
which is consistent with both A and B being knaves. We can conclude that both A and B are 
knaves. < 


We pose more of Smullyan’s puzzles about knights and knaves in Exercises 23-27. In Ex- 
ercises 28-35 we introduce related puzzles where we have three types of people, knights and 
knaves as in this puzzle together with spies who can lie. 

Next, we pose a puzzle known as the muddy children puzzle for the case of two children. 


A father tells his two children, a boy and a girl, to play in their backyard without getting dirty. 
However, while playing, both children get mud on their foreheads. When the children stop play- 
ing, the father says “At least one of you has a muddy forehead,” and then asks the children to 
answer “Yes” or “No” to the question: “Do you know whether you have a muddy forehead?” The 
father asks this question twice. What will the children answer each time this question is asked, 
assuming that a child can see whether his or her sibling has a muddy forehead, but cannot see 
his or her own forehead? Assume that both children are honest and that the children answer each 
question simultaneously. 


Solution: Let s be the statement that the son has a muddy forehead and let d be the statement that 
the daughter has a muddy forehead. When the father says that at least one of the two children 
has a muddy forehead, he is stating that the disjunction s V d is true. Both children will answer 
“No” the first time the question is asked because each sees mud on the other child’s forehead. 
That is, the son knows that d is true, but does not know whether s is true, and the daughter 
knows that s is true, but does not know whether d is true. 

After the son has answered “No” to the first question, the daughter can determine that d 
must be true. This follows because when the first question is asked, the son knows that s V d is 
true, but cannot determine whether s is true. Using this information, the daughter can conclude 
that d must be true, for if d were false, the son could have reasoned that because s V d is true, 


RAYMOND SMULLYAN (1919-2017) Raymond Smullyan, the son of a businessman and a homemaker, was 
born in Far Rockaway, Queens, New York. He dropped out of high school because he wanted to study what he 
was really interested in and not standard high school material. After attending Pacific College and Reed College 
in Oregon, he earned an undergraduate degree in mathematics at the University of Chicago in 1955. He paid his 
college expenses by performing magic tricks at parties and clubs, using the stage name Five-Ace Merrill. He 
obtained a Ph.D. in logic in 1959 at Princeton, studying under Alonzo Church. After graduating from Princeton, 
he taught mathematics and logic at Dartmouth College, Princeton University, Yeshiva University, and the City 
University of New York. He joined the philosophy department at Indiana University in 1981, where be became 
an emeritus professor. 

Smullyan wrote many books on recreational logic and mathematics, including Satan, Cantor, and Infin- 
ity; What Is the Name of This Book?; The Lady or the Tiger?; Alice in Puzzleland; To Mock a Mockingbird; 


Forever Undecided; and The Riddle of Scheherazade: Amazing Logic Puzzles, Ancient and Modern. Because his logic puzzles 
are challenging, entertaining, and thought-provoking, he was considered to be a modern-day Lewis Carroll. Smullyan also wrote 
several books about the application of deductive logic to chess, three collections of philosophical essays and aphorisms, and several 
advanced books on mathematical logic and set theory. He was particularly interested in self-reference and worked on extending 
some of Gédel’s results that show that it is impossible to write a computer program that can solve all mathematical problems. He 
was also particularly interested in explaining ideas from mathematical logic to the public. 

Smullyan was a talented musician and often played piano with his second wife, who was a concert-level pianist. Making 
telescopes was one of his hobbies and he was interested in optics and stereo photography. He said, “I’ve never had a conflict between 
teaching and research as some people do because when I’m teaching, I’m doing research.” Smullyan is the subject of a documentary 
short film entitled This Film Needs No Title. 
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In Chapter 12 we will 
design some useful 
circuits. 


EXAMPLE 10 


then s must be true, and he would have answered “Yes” to the first question. The son can reason 
in a similar way to determine that s must be true. It follows that both children answer “Yes” the 
second time the question is asked. < 


1.2.6 Logic Circuits 


Propositional logic can be applied to the design of computer hardware. This was first observed 
in 1938 by Claude Shannon in his MIT master’s thesis. In Chapter 12 we will study this topic 
in depth. (See that chapter for a biography of Shannon.) We give a brief introduction to this 
application here. 

A logic circuit (or digital circuit) receives input signals p,, p>, ...,p,,, each a bit [either 
O (off) or 1 (on)], and produces output signals s,, 55, ...,5,, each a bit. In this section we will 
restrict our attention to logic circuits with a single output signal; in general, digital circuits may 
have multiple outputs. 

Complicated digital circuits can be constructed from three basic circuits, called gates, 
shown in Figure 1. The inverter, or NOT gate, takes an input bit p, and produces as output 
ap. The OR gate takes two input signals p and q, each a bit, and produces as output the signal 
p V q. Finally, the AND gate takes two input signals p and q, each a bit, and produces as out- 
put the signal p A q. We use combinations of these three basic gates to build more complicated 
circuits, such as that shown in Figure 2. 


5 È =p p PVG p p^q 
I—» q 


Inverter OR gate AND gate 
FIGURE 1 Basic logic gates. 


Given a circuit built from the basic logic gates and the inputs to the circuit, we determine 
the output by tracing through the circuit, as Example 10 shows. 


Determine the output for the combinatorial circuit in Figure 2. 


Solution: In Figure 2 we display the output of each logic gate in the circuit. We see that the 
AND gate takes input of p and ~q, the output of the inverter with input q, and produces p A 
aq. Next, we note that the OR gate takes input p A ~q and ~r, the output of the inverter with 
input r, and produces the final output (p A ~q) V ar. < 


Suppose that we have a formula for the output of a digital circuit in terms of negations, dis- 
junctions, and conjunctions. Then, we can systematically build a digital circuit with the desired 
output, as illustrated in Example 11. 


P pA~| 
> 


FIGURE 2 A combinatorial circuit. 
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FIGURE3 The circuit for (p V =r) A (=p V (q V `r)). 


Build a digital circuit that produces the output (p V =r) A (=p V (q V 7r)) when given input bits 


Solution: To construct the desired circuit, we build separate circuits for p V ar and for ap V (q V 
ar) and combine them using an AND gate. To construct a circuit for p V =r, we use an inverter 
to produce =r from the input r. Then, we use an OR gate to combine p and ~r. To build a circuit 
for =p V (q V ar), we first use an inverter to obtain ~ar. Then we use an OR gate with inputs q 
and =r to obtain q V ^r. Finally, we use another inverter and an OR gate to get ap V (q V `r) 


To complete the construction, we employ a final AND gate, with inputs p V =r and ap V 
(q V ar). The resulting circuit is displayed in Figure 3. < 


We will study logic circuits in great detail in Chapter 12 in the context of Boolean algebra, 


EXAMPLE 11 
p, q, andr. 
from the inputs p and q V ~r. 
and with different notation. 
Exercises 


In Exercises 1—6, translate the given statement into proposi- 
tional logic using the propositions provided. 


1. 


You cannot edit a protected Wikipedia entry unless you 
are an administrator. Express your answer in terms of e: 
“You can edit a protected Wikipedia entry” and a: “You 
are an administrator.” 


. You can see the movie only if you are over 18 years old 


or you have the permission of a parent. Express your an- 
swer in terms of m: “You can see the movie,” e: “You are 
over 18 years old,” and p: “You have the permission of a 
parent.” 


. You can graduate only if you have completed the require- 


ments of your major and you do not owe money to the 
university and you do not have an overdue library book. 
Express your answer in terms of g: “You can graduate,” 
m: “You owe money to the university,” r: “You have com- 
pleted the requirements of your major,” and b: “You have 
an overdue library book.” 


. To use the wireless network in the airport you must pay 


the daily fee unless you are a subscriber to the service. 
Express your answer in terms of w: “You can use the 
wireless network in the airport,” d: “You pay the daily 
fee,” and s: “You are a subscriber to the service.” 


. You are eligible to be President of the U.S.A. only if you 


are at least 35 years old, were born in the U.S.A., or at the 


time of your birth both of your parents were citizens, and 
you have lived at least 14 years in the country. Express 
your answer in terms of e: “You are eligible to be Pres- 
ident of the U.S.A.,” a: “You are at least 35 years old,” 
b: “You were born in the U.S.A.,” p: “At the time of your 
birth, both of your parents were citizens,” and r: “You 
have lived at least 14 years in the U.S.A.” 


. You can upgrade your operating system only if you have a 


32-bit processor running at 1 GHz or faster, at least 1 GB 
RAM, and 16 GB free hard disk space, or a 64-bit pro- 
cessor running at 2 GHz or faster, at least 2 GB RAM, 
and at least 32 GB free hard disk space. Express your an- 
swer in terms of u: “You can upgrade your operating sys- 
tem,” b33: “You have a 32-bit processor,” b¢4: “You have 
a 64-bit processor,” g,: “Your processor runs at 1 GHz or 
faster,” g,: “Your processor runs at 2 GHz or faster,” r}: 
“Your processor has at least 1 GB RAM,” r,: “Your pro- 
cessor has at least 2 GB RAM,” hs: “You have at least 
16 GB free hard disk space,” and h3,: “You have at least 
32 GB free hard disk space.” 


7. Express these system specifications using the proposi- 


tions p: “The message is scanned for viruses” and q: “The 

message was sent from an unknown system” together 

with logical connectives (including negations). 

a) “The message is scanned for viruses whenever the 
message was sent from an unknown system.” 
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b) “The message was sent from an unknown system but 
it was not scanned for viruses.” 

c) “It is necessary to scan the message for viruses when- 
ever it was sent from an unknown system.” 

d) “When a message is not sent from an unknown sys- 
tem it is not scanned for viruses.” 


. Express these system specifications using the proposi- 


tions p: “The user enters a valid password,” g: “Access 

is granted,” and r: “The user has paid the subscription 

fee” and logical connectives (including negations). 

a) “The user has paid the subscription fee, but does not 
enter a valid password.” 

b) “Access is granted whenever the user has paid the 
subscription fee and enters a valid password.” 

c) “Access is denied if the user has not paid the subscrip- 
tion fee.” 

d) “If the user has not entered a valid password but has 
paid the subscription fee, then access is granted.” 


. Are these system specifications consistent? “The system 


is in multiuser state if and only if it is operating normally. 
If the system is operating normally, the kernel is func- 
tioning. The kernel is not functioning or the system is in 
interrupt mode. If the system is not in multiuser state, 
then it is in interrupt mode. The system is not in interrupt 
mode.” 


Are these system specifications consistent? “Whenever 
the system software is being upgraded, users cannot ac- 
cess the file system. If users can access the file system, 
then they can save new files. If users cannot save new 
files, then the system software is not being upgraded.” 


Are these system specifications consistent? “The router 
can send packets to the edge system only if it supports the 
new address space. For the router to support the new ad- 
dress space it is necessary that the latest software release 
be installed. The router can send packets to the edge sys- 
tem if the latest software release is installed. The router 
does not support the new address space.” 


Are these system specifications consistent? “If the file 
system is not locked, then new messages will be queued. 
If the file system is not locked, then the system is func- 
tioning normally, and conversely. If new messages are not 
queued, then they will be sent to the message buffer. If the 
file system is not locked, then new messages will be sent 
to the message buffer. New messages will not be sent to 
the message buffer.” 


What Boolean search would you use to look for Web 
pages about beaches in New Jersey? What if you wanted 
to find Web pages about beaches on the isle of Jersey (in 
the English Channel)? 


What Boolean search would you use to look for Web 
pages about hiking in West Virginia? What if you wanted 
to find Web pages about hiking in Virginia, but not in 
West Virginia? 

What Google search would you use to look for Web pages 
relating to Ethiopian restaurants in New York or New 
Jersey? 
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. What Google search would you use to look for men’s 
shoes or boots not designed for work? 


. Suppose that in Example 7, the inscriptions on Trunks 1, 
2, and 3 are “The treasure is in Trunk 3,” “The treasure is 
in Trunk 1,” and “This trunk is empty.” For each of these 
statements, determine whether the Queen who never 
lies could state this, and if so, which trunk the treasure 
is in. 

a) “All the inscriptions are false.” 

b) “Exactly one of the inscriptions is true.” 
c) “Exactly two of the inscriptions are true.” 
d) “All three inscriptions are true.” 


. Suppose that in Example 7 there are treasures in two of 
the three trunks. The inscriptions on Trunks 1, 2, and 3 
are “This trunk is empty,” “There is a treasure in Trunk 
1,” and “There is a treasure in Trunk 2.” For each of these 
statements, determine whether the Queen who never lies 
could state this, and if so, which two trunks the treasures 
are in. 

a) “All the inscriptions are false.” 

b) “Exactly one of the inscriptions is true.” 

c) “Exactly two of the inscriptions are true.” 

d) “All three inscriptions are true.” 

Each inhabitant of a remote village always tells the truth 
or always lies. A villager will give only a “Yes” or a “No” 
response to a question a tourist asks. Suppose you are a 
tourist visiting this area and come to a fork in the road. 
One branch leads to the ruins you want to visit; the other 
branch leads deep into the jungle. A villager is standing 
at the fork in the road. What one question can you ask the 
villager to determine which branch to take? 


Anexplorer is captured by a group of cannibals. There are 
two types of cannibals—those who always tell the truth 
and those who always lie. The cannibals will barbecue 
the explorer unless he can determine whether a particu- 
lar cannibal always lies or always tells the truth. He is 
allowed to ask the cannibal exactly one question. 

a) Explain why the question “Are you a liar?” does not 
work. 

b) Find a question that the explorer can use to determine 
whether the cannibal always lies or always tells the 
truth. 

When three professors are seated in a restaurant, the host- 

ess asks them: “Does everyone want coffee?” The first 

professor says “I do not know.” The second professor then 
says “I do not know.” Finally, the third professor says 

“No, not everyone wants coffee.” The hostess comes back 

and gives coffee to the professors who want it. How did 

she figure out who wanted coffee? 


When planning a party you want to know whom to in- 
vite. Among the people you would like to invite are three 
touchy friends. You know that if Jasmine attends, she will 
become unhappy if Samir is there, Samir will attend only 
if Kanti will be there, and Kanti will not attend unless 
Jasmine also does. Which combinations of these three 
friends can you invite so as not to make someone un- 


happy? 


Exercises 23-27 relate to inhabitants of the island of knights 

and knaves created by Smullyan, where knights always tell the 

truth and knaves always lie. You encounter two people, A and 

B. Determine, if possible, what A and B are if they address you 

in the ways described. If you cannot determine what these two 

people are, can you draw any conclusions? 

23. A says “At least one of us is a knave” and B says nothing. 

24. A says “The two of us are both knights” and B says “A is 
a knave.” 

25. A says “I am a knave or B is a knight” and B says nothing. 
26. Both A and B say “I am a knight.” 
27. A says “We are both knaves” and B says nothing. 
Exercises 28-35 relate to inhabitants of an island on which 
there are three kinds of people: knights who always tell the 
truth, knaves who always lie, and spies (called normals by 
Smullyan [Sm78]) who can either lie or tell the truth. You 
encounter three people, A, B, and C. You know one of these 
people is a knight, one is a knave, and one is a spy. Each of the 
three people knows the type of person each of other two is. For 
each of these situations, if possible, determine whether there 
is a unique solution and determine who the knave, knight, and 
spy are. When there is no unique solution, list all possible so- 
lutions or state that there are no solutions. 

28. A says “C is the knave,” B says “A is the knight,” and C 
says “I am the spy.” 

29. A says “I am the knight,” B says “I am the knave,” and C 
says “B is the knight.” 

30. A says “I am the knave,” B says “I am the knave,” and C 
says “I am the knave.” 

31. A says “I am the knight,” B says “A is telling the truth,” 
and C says “I am the spy.” 

32. A says “I am the knight,” B says “A is not the knave,” and 
C says “B is not the knave.” 

33. A says “I am the knight,” B says “I am the knight,” and C 
says “I am the knight.” 

34. A says “I am not the spy,” B says “I am not the spy,” and 
C says “A is the spy.” 

35. A says “I am not the spy,” B says “I am not the spy,” and 
C says “I am not the spy.” 

Exercises 36—42 are puzzles that can be solved by translating 

statements into logical expressions and reasoning from these 

expressions using truth tables. 

36. The police have three suspects for the murder of Mr. 
Cooper: Mr. Smith, Mr. Jones, and Mr. Williams. Smith, 
Jones, and Williams each declare that they did not kill 
Cooper. Smith also states that Cooper was a friend of 
Jones and that Williams disliked him. Jones also states 
that he did not know Cooper and that he was out of town 
the day Cooper was killed. Williams also states that he 
saw both Smith and Jones with Cooper the day of the 
killing and that either Smith or Jones must have killed 
him. Can you determine who the murderer was if 
a) one of the three men is guilty, the two innocent men 

are telling the truth, but the statements of the guilty 
man may or may not be true? 
b) innocent men do not lie? 
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37. Steve would like to determine the relative salaries of three 
coworkers using two facts. First, he knows that if Fred 
is not the highest paid of the three, then Janice is. Sec- 
ond, he knows that if Janice is not the lowest paid, then 
Maggie is paid the most. Is it possible to determine the 
relative salaries of Fred, Maggie, and Janice from what 
Steve knows? If so, who is paid the most and who the 
least? Explain your reasoning. 


38. Five friends have access to a chat room. Is it possible to 
determine who is chatting if the following information is 
known? Either Kevin or Heather, or both, are chatting. 
Either Randy or Vijay, but not both, are chatting. If Abby 
is chatting, so is Randy. Vijay and Kevin are either both 
chatting or neither is. If Heather is chatting, then so are 
Abby and Kevin. Explain your reasoning. 


39. A detective has interviewed four witnesses to a crime. 
From the stories of the witnesses the detective has con- 
cluded that if the butler is telling the truth then so is the 
cook; the cook and the gardener cannot both be telling 
the truth; the gardener and the handyman are not both ly- 
ing; and if the handyman is telling the truth then the cook 
is lying. For each of the four witnesses, can the detec- 
tive determine whether that person is telling the truth or 
lying? Explain your reasoning. 

40. Four friends have been identified as suspects for an unau- 
thorized access into a computer system. They have made 
statements to the investigating authorities. Alice said, 
“Carlos did it.” John said, “I did not do it.” Carlos said, 
“Diana did it.” Diana said, “Carlos lied when he said that 
I did it.” 

a) If the authorities also know that exactly one of the 
four suspects is telling the truth, who did it? Explain 
your reasoning. 

b) Ifthe authorities also know that exactly one is lying, 
who did it? Explain your reasoning. 

41. Suppose there are signs on the doors to two rooms. The 
sign on the first door reads “In this room there is a lady, 
and in the other one there is a tiger”; and the sign on the 
second door reads “In one of these rooms, there is a lady, 
and in one of them there is a tiger.” Suppose that you 
know that one of these signs is true and the other is false. 
Behind which door is the lady? 


* 42. Solve this famous logic puzzle, attributed to Albert Ein- 
stein, and known as the zebra puzzle. Five men with 


Links » different nationalities and with different jobs live in con- 
i 


secutive houses on a street. These houses are painted 
different colors. The men have different pets and have dif- 
ferent favorite drinks. Determine who owns a zebra and 
whose favorite drink is mineral water (which is one of the 
favorite drinks) given these clues: The Englishman lives 
in the red house. The Spaniard owns a dog. The Japanese 
man is a painter. The Italian drinks tea. The Norwegian 
lives in the first house on the left. The green house is im- 
mediately to the right of the white one. The photographer 
breeds snails. The diplomat lives in the yellow house. 
Milk is drunk in the middle house. The owner of the green 
house drinks coffee. The Norwegian’s house is next to the 
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blue one. The violinist drinks orange juice. The fox is in 
a house next to that of the physician. The horse is in a 
house next to that of the diplomat. [Hint: Make a table 
where the rows represent the men and columns represent 
the color of their houses, their jobs, their pets, and their 
favorite drinks and use logical reasoning to determine the 
correct entries in the table.] 

Freedonia has 50 senators. Each senator is either honest 
or corrupt. Suppose you know that at least one of the 
Freedonian senators is honest and that, given any two 
Freedonian senators, at least one is corrupt. Based on these 
facts, can you determine how many Freedonian senators are 
honest and how many are corrupt? If so, what is the answer? 
Find the output of each of these combinatorial circuits. 


a) p >> 
1 —>Mo 


Dey Da 


b) 


q 


45. 


46. 


47. 


Find the output of each of these combinatorial circuits. 


e a 


a) P 


ee 
aso D 


Construct a combinatorial circuit using inverters, 
OR gates, and AND gates that produces the output 
(p Anr) V (7q A r) from input bits p, q, and r. 


Construct a combinatorial circuit using inverters, 
OR gates, and AND gates that produces the output 
(py =r) A7q) V (=p A^ (q V r)) from input bits p, q, 
and r. 


Propositional Equivalences 


Definition 1 


EXAMPLE 1 


1.3.1 Introduction 


An important type of step used in a mathematical argument is the replacement of a statement 
with another statement with the same truth value. Because of this, methods that produce propo- 
sitions with the same truth value as a given compound proposition are used extensively in the 
construction of mathematical arguments. Note that we will use the term “compound proposi- 
tion” to refer to an expression formed from propositional variables using logical operators, such 
as p ^q. 

We begin our discussion with a classification of compound propositions according to their 
possible truth values. 


A compound proposition that is always true, no matter what the truth values of the proposi- 
tional variables that occur in it, is called a tautology. A compound proposition that is always 
false is called a contradiction. A compound proposition that is neither a tautology nor a con- 
tradiction is called a contingency. 


Tautologies and contradictions are often important in mathematical reasoning. Example 1 illus- 
trates these types of compound propositions. 


We can construct examples of tautologies and contradictions using just one propositional vari- 
able. Consider the truth tables of p V =p and p A =p, shown in Table 1. Because p V =p is always 
true, it is a tautology. Because p A =p is always false, it is a contradiction. < 


Demo » 


Definition 2 


Extra » 
Examples 


EXAMPLE 2 
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TABLE 1 Examples of a Tautology 
and a Contradiction. 


1.3.2 Logical Equivalences 


Compound propositions that have the same truth values in all possible cases are called logically 
equivalent. We can also define this notion as follows. 


The compound propositions p and q are called logically equivalent if p + q is a tautology. 
The notation p = q denotes that p and q are logically equivalent. 


Remark: The symbol = is not a logical connective, and p = q is not a compound proposition 
but rather is the statement that p + q is a tautology. The symbol = is sometimes used instead 
of = to denote logical equivalence. 


One way to determine whether two compound propositions are equivalent is to use a truth 
table. In particular, the compound propositions p and q are equivalent if and only if the columns 
giving their truth values agree. Example 2 illustrates this method to establish an extremely 
important and useful logical equivalence, namely, that of =(p V q) with =p A ~q. This logi- 
cal equivalence is one of the two De Morgan laws, shown in Table 2, named after the English 
mathematician Augustus De Morgan, of the mid-nineteenth century. 


TABLE 2 De 
Morgan’s Laws. 


Pph) = pv q 
“(pV q) = =p ^q 


Show that -(p V q) and ap A ~q are logically equivalent. 


Solution: The truth tables for these compound propositions are displayed in Table 3. Because 
the truth values of the compound propositions =(p V q) and sp A ~q agree for all possible com- 
binations of the truth values of p and q, it follows that =(p V q) e (ap A ^q) is a tautology and 
that these compound propositions are logically equivalent. < 


TABLE 3 Truth Tables for =@ V q) and ~p A ~q. 


a i evo | æ | œ | wam | 


F 
F 
F 
T 
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EXAMPLE 3 


EXAMPLE 4 


The next example establishes an extremely important equivalence. It allows us to replace 
conditional statements with negations and disjunctions. 


Show that p > q and =p V q are logically equivalent. (This is known as the conditional- 
disjunction equivalence.) 


Solution: We construct the truth table for these compound propositions in Table 4. Because the 
truth values of =p V q and p > q agree, they are logically equivalent. < 


TABLE 4 Truth Tables for =p V q and 
p >q. 


We will now establish a logical equivalence of two compound propositions involving three 
different propositional variables p, q, and r. To use a truth table to establish such a logical 
equivalence, we need eight rows, one for each possible combination of truth values of these 
three variables. We symbolically represent these combinations by listing the truth values of p, 
q, and r, respectively. These eight combinations of truth values are TTT, TTF, TFT, TFF, FTT, 
FTF, FFT, and FFF; we use this order when we display the rows of the truth table. Note that we 
need to double the number of rows in the truth tables we use to show that compound propositions 
are equivalent for each additional propositional variable, so that 16 rows are needed to establish 
the logical equivalence of two compound propositions involving four propositional variables, 
and so on. In general, 2” rows are required if a compound proposition involves n propositional 
variables. Because of the rapid growth of 2”, more efficient ways are needed to establish logical 
equivalences, such as by using ones we already know. This technique will be discussed later. 


Show that p V (q A r) and (p V q) A (p V r) are logically equivalent. This is the distributive law 
of disjunction over conjunction. 


Solution: We construct the truth table for these compound propositions in Table 5. Because the 


truth values of p V (q A r) and (p V q) A (p V r) agree, these compound propositions are logically 
equivalent. < 


TABLE 5 A Demonstration That p V (q Ar) and (p V q) A (p V r) Are Logically 


Equivalent 


s 
Q 
`~ 


T 
T 
T 
T 
F 
F 
F 
F 


amaan 
mamamana 
co ir. aise aie s e AS 
DMemeaAaaaAa 
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TABLE 6 Logical Equivalences. 


PAT=p Identity laws 
pPVF=p 


Domination laws 


Idempotent laws 


Double negation law 


Commutative laws 


(PVQVr=pVv(qvr) Associative laws 
(PAQAr=pPACANr) 


PVGAN=(PVQAMV?N) Distributive laws 
PACVT)=(DAQV@PADN) 


(pq) = pV 7q De Morgan’s laws 
“PVM =p rq 

PVPAQ)=P Absorption laws 
PAPVYQ=P 


Negation laws 


The identities in Table 6 Table 6 contains some important equivalences. In these equivalences, T denotes the com- 
me — or pound proposition that is always true and F denotes the compound proposition that is always 
SNEER NEA Tabe false. We also display some useful equivalences for compound propositions involving condi- 
5 of Section 12.1. See tional statements and biconditional statements in Tables 7 and 8, respectively. The reader is 
Table 1 in Section 2.2 asked to verify the equivalences in Tables 6-8 in the exercises. 


for analogous set 
identities. 


TABLE 7 Logical Equivalences TABLE 8 Logical 
Involving Conditional Equivalences Involving 
Statements. Biconditional Statements. 
PpP>4=@pVq peq=p>9) ^4 >p) 
a e > 7p peq=7p°-7q 
pPYq=yp>4q POqd=(PAQV(PA79) 


pPAq=-—) > ~q) (p 2q)=p e -4q 


Pp > gq) =pA-~q 

(p> QAPrr) =p > (Ar) 
P= => rE pyaar 
p>)yp>r)=p> 4yr) 
PrnVgqrn=PAgrr 
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Be careful not to apply 
logical identities, such 
as associative laws, 
distributive laws, or De 
Morgan’s laws, to 
expressions that have a 
mix of conjunctions and 
disjunctions when the 
identities only apply 
when all these operators 
are the same. 


When using De 
Morgan’s laws, 
remember to change 
the logical connective 
after you negate. 


EXAMPLE 5 


Assessment » 


The associative law for disjunction shows that the expression p V q V r is well defined, in 
the sense that it does not matter whether we first take the disjunction of p with g and then 
the disjunction of p V q with r, or if we first take the disjunction of q and r and then take the 
disjunction of p with q V r. Similarly, the expression p A q A r is well defined. By extending 
this reasoning, it follows that p; V pə V + V p, and p; A py A ++: A p, are well defined whenever 
Pi Po +++» Pn are propositions. 

Furthermore, note that De Morgan’s laws extend to 


(P1 V Pa V+ V Pa) = OPi ATP A ATPa) 


and 


(Pi AP Av A Pa) = OPi V P2 Vi V AP): 


We will sometimes use the notation Vi p; for pi Vp V Vp, and Nja p; for 
Pi AP2 A+: Ap,. Using this notation, the extended version of De Morgan’s laws can 
be written concisely as =( View p;) = N;-1 7P; and | Nii p;) = V1 7p; (Methods for 
proving these identities will be given in Section 5.1.) 

A truth table with 2” rows is needed to prove the equivalence of two compound propositions 
in n variables. (Note that the number of rows doubles for each additional propositional variable 
added. See Chapter 6 for details about solving counting problems such as this.) Because 2” 
grows extremely rapidly as n increases (see Section 3.2), the use of truth tables to establish 
equivalences becomes impractical as the number of variables grows. It is quicker to use other 
methods, such as employing logical equivalences that we already know. How that can be done 
is discussed later in this section. 


1.3.3 Using De Morgan’s Laws 


The two logical equivalences known as De Morgan’s laws are particularly important. They tell 
us how to negate conjunctions and how to negate disjunctions. In particular, the equivalence 
~(p V q) = =p ^ ~q tells us that the negation of a disjunction is formed by taking the conjunction 
of the negations of the component propositions. Similarly, the equivalence ~(p A q) = =p V ~q 
tells us that the negation of a conjunction is formed by taking the disjunction of the negations 
of the component propositions. Example 5 illustrates the use of De Morgan’s laws. 


Use De Morgan’s laws to express the negations of “Miguel has a cellphone and he has a laptop 
computer” and “Heather will go to the concert or Steve will go to the concert.” 


Solution: Let p be “Miguel has a cellphone” and q be “Miguel has a laptop computer.” Then 
“Miguel has a cellphone and he has a laptop computer” can be represented by p A q. By the 
first of De Morgan’s laws, ~(p A q) is equivalent to =p V ~q. Consequently, we can express the 
negation of our original statement as “Miguel does not have a cellphone or he does not have a 
laptop computer.” 

Let r be “Heather will go to the concert” and s be “Steve will go to the concert.” Then 
“Heather will go to the concert or Steve will go to the concert” can be represented by r V s. By 
the second of De Morgan’s laws, ~(r V s) is equivalent to =r A ss. Consequently, we can express 
the negation of our original statement as “Heather will not go to the concert and Steve will not 
go to the concert.” < 
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1.3.4 Constructing New Logical Equivalences 


The logical equivalences in Table 6, as well as any others that have been established (such as 
those shown in Tables 7 and 8), can be used to construct additional logical equivalences. The 
reason for this is that a proposition in a compound proposition can be replaced by a compound 
proposition that is logically equivalent to it without changing the truth value of the original 
compound proposition. This technique is illustrated in Examples 6-8, where we also use the 
fact that if p and q are logically equivalent and q and r are logically equivalent, then p and r are 
logically equivalent (see Exercise 60). 


EXAMPLE 6 Show that ~(p > q) and p A 7¢ are logically equivalent. 


ae » Solution: We could use a truth table to show that these compound propositions are equivalent 
(similar to what we did in Example 4). Indeed, it would not be hard to do so. However, we want 
to illustrate how to use logical identities that we already know to establish new logical identities, 
something that is of practical importance for establishing equivalences of compound proposi- 
tions with a large number of variables. So, we will establish this equivalence by developing a 
series of logical equivalences, using one of the equivalences in Table 6 at a time, starting with 
~(p > q) and ending with p A aq. We have the following equivalences. 


a(p > g) =7C7p V gq) by the conditional-disjunction equivalence (Example 3) 
=-(7p) A7q__ by the second De Morgan law 
=pA-7q by the double negation law 


EXAMPLE 7 Show that -(p Vv (p A q)) and =p A ~q are logically equivalent by developing a series of logical 
equivalences. 


Solution: We will use one of the equivalences in Table 6 at a time, starting with =(p V (=p A q)) 
and ending with ~p A ^q. (Note: we could also easily establish this equivalence using a truth 
table.) We have the following equivalences. 


AUGUSTUS DE MORGAN (1806-1871) Augustus De Morgan was born in India, where his father was a 
colonel in the Indian army. De Morgan’s family moved to England when he was 7 months old. He attended 
private schools, where in his early teens he developed a strong interest in mathematics. De Morgan studied 
at Trinity College, Cambridge, graduating in 1827. Although he considered medicine or law, he decided on 
mathematics for his career. He won a position at University College, London, in 1828, but resigned after the 
college dismissed a fellow professor without giving reasons. However, he resumed this position in 1836 when 
his successor died, remaining until 1866. 

De Morgan was a noted teacher who stressed principles over techniques. His students in- 

©Bettmann/Gerty Images cluded many famous mathematicians, including Augusta Ada, Countess of Lovelace, who was Charles 

Babbage’s collaborator in his work on computing machines (see page 32 for biographical notes 
on Augusta Ada). (De Morgan cautioned the countess against studying too much mathematics, because it might interfere with her 
childbearing abilities!) 

De Morgan was an extremely prolific writer, publishing more than 1000 articles in more than 15 periodicals. De Morgan also 
wrote textbooks on many subjects, including logic, probability, calculus, and algebra. In 1838 he presented what was perhaps the first 
clear explanation of an important proof technique known as mathematical induction (discussed in Section 5.1 of this text), a term 
he coined. In the 1840s De Morgan made fundamental contributions to the development of symbolic logic. He invented notations 
that helped him prove propositional equivalences, such as the laws that are named after him. In 1842 De Morgan presented what 
is considered to be the first precise definition of a limit and developed new tests for convergence of infinite series. De Morgan was 
also interested in the history of mathematics and wrote biographies of Newton and Halley. 

In 1837 De Morgan married Sophia Frend, who wrote his biography in 1882. De Morgan’s research, writing, and teaching left 
little time for his family or social life. Nevertheless, he was noted for his kindness, humor, and wide range of knowledge. 
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apy (=p A q)) = 3p rnp Aq) by the second De Morgan law 
= ap A[7(7p) V ~q] by the first De Morgan law 
= =p A(pV 7q) by the double negation law 
= (pAp)V Cp Ang) bythe second distributive law 
=Fv (pang) because =p Ap = F 
= (ap A7q)VF by the commutative law for disjunction 
=—pA7q by the identity law for F 

Consequently =(p V (=p A q)) and =p A ~q are logically equivalent. < 


EXAMPLE 8 Show that (p A q) > (p V q) is a tautology. 


Solution: To show that this statement is a tautology, we will use logical equivalences to demon- 
strate that it is logically equivalent to T. (Note: This could also be done using a truth table.) 


PAD? PVDN=7PAQDVOEV| by Example 3 
(=p V 7q) V (Pp V q) by the first De Morgan law 
= (7p V p) V (79qV q) | by the associative and commutative 


laws for disjunction 


=TvT by Example 1 and the commutative 
law for disjunction 


T by the domination law < 


AUGUSTA ADA, COUNTESS OF LOVELACE (1815-1852) Augusta Ada was the only child from the mar- 
riage of the flamboyant and notorious poet Lord Byron and Lady Byron, Annabella Millbanke, who separated 
when Ada was 1 month old, because of Lord Byron’s scandalous affair with his half sister. The Lord Byron 
had quite a reputation, being described by one of his lovers as “mad, bad, and dangerous to know.” Lady Byron 
was noted for her intellect and had a passion for mathematics; she was called by Lord Byron “The Princess of 
Parallelograms.” Augusta was raised by her mother, who encouraged her intellectual talents especially in music 
and mathematics, to counter what Lady Byron considered dangerous poetic tendencies. At this time, women 
è were not allowed to attend universities and could not join learned societies. Nevertheless, Augusta pursued 
©Hulton Archive/Getty her mathematical studies independently and with mathematicians, including William Frend. She was also en- 
Images couraged by another female mathematician, Mary Somerville, and in 1834 at a dinner party hosted by Mary 

Somerville, she learned about Charles Babbage’s ideas for a calculating machine, called the Analytic Engine. 

In 1838 Augusta Ada married Lord King, later elevated to Earl of Lovelace. Together they had three children. 

Augusta Ada continued her mathematical studies after her marriage. Charles Babbage had continued work on his Analytic 
Engine and lectured on this in Europe. In 1842 Babbage asked Augusta Ada to translate an article in French describing Babbage’s 
invention. When Babbage saw her translation, he suggested she add her own notes, and the resulting work was three times the 
length of the original. The most complete accounts of the Analytic Engine are found in Augusta Ada’s notes. In her notes, she 
compared the working of the Analytic Engine to that of the Jacquard loom, with Babbage’s punch cards analogous to the cards used 
to create patterns on the loom. Furthermore, she recognized the promise of the machine as a general purpose computer much better 
than Babbage did. She stated that the “engine is the material expression of any indefinite function of any degree of generality and 
complexity.” Her notes on the Analytic Engine anticipate many future developments, including computer-generated music. Augusta 
Ada published her writings under her initials A.A.L., concealing her identity as a woman as did many women at a time when women 
were not considered to be the intellectual equals of men. After 1845 she and Babbage worked toward the development of a system 
to predict horse races. Unfortunately, their system did not work well, leaving Augusta Ada heavily in debt at the time of her death 
at an unfortunately young age from uterine cancer. 

In 1953 Augusta Ada’s notes on the Analytic Engine were republished more than 100 years after they were written, and after 
they had been long forgotten. In his work in the 1950s on the capacity of computers to think (and his influential Turing test for 
determining whether a machine is intelligent), Alan Turing responded to Augusta Ada’s statement that “The Analytic Engine has 
no pretensions whatever to originate anything. It can do whatever we know how to order it to perform.” This “dialogue” between 
Turing and Augusta Ada is still the subject of controversy. Because of her fundamental contributions to computing, the programming 
language Ada is named in honor of the Countess of Lovelace. 
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1.3.5 Satisfiability 


A compound proposition is satisfiable if there is an assignment of truth values to its variables 
that makes it true (that is, when it is a tautology or a contingency). When no such assignments 
exists, that is, when the compound proposition is false for all assignments of truth values to 
its variables, the compound proposition is unsatisfiable. Note that a compound proposition is 
unsatisfiable if and only if its negation is true for all assignments of truth values to the variables, 
that is, if and only if its negation is a tautology. 

When we find a particular assignment of truth values that makes a compound proposition 
true, we have shown that it is satisfiable; such an assignment is called a solution of this particular 
satisfiability problem. However, to show that a compound proposition is unsatisfiable, we need 
to show that every assignment of truth values to its variables makes it false. Although we can 
always use a truth table to determine whether a compound proposition is satisfiable, it is often 
more efficient not to, as Example 9 demonstrates. 


Determine whether each of the compound propositions (pV7q)A(qV7r)A 
(r V 7p), (PVqgVrnyACpV7qV7r), and (pV 7g) Alq yar) Ary ap) A 
(pV qVr) A (=p V7q V 771) is satisfiable. 


Solution: Instead of using a truth table to solve this problem, we will reason about truth values. 
Note that (p V =q) A (q V ar) A (r V =p) is true when the three variables p, q, and r have the 
same truth value (see Exercise 42 of Section 1.1). Hence, it is satisfiable as there is at least 
one assignment of truth values for p, q, and r that makes it true. Similarly, note that (p V q V 
r) A p V ~q V `r) is true when at least one of p, q, and r is true and at least one is false (see 
Exercise 43 of Section 1.1). Hence, (p V q V r) A (=p V ~q V 71) is satisfiable, as there is at least 
one assignment of truth values for p, q, and r that makes it true. 

Finally, note that for (p V 7g) A(qV ar) A(rV ap) A(pV q Vr) A(Ap V 7g V ar) to be 
true, (p Vaq) A(qV ar) A(rV ap) and (pV qVr)A (=p V-7qV =r) must both be true. For 
the first to be true, the three variables must have the same truth values, and for the sec- 
ond to be true, at least one of the three variables must be true and at least one must be 
false. However, these conditions are contradictory. From these observations we conclude 
that no assignment of truth values to p, q, and r makes (p V aq) A(qgV7r) A(rV ap) A 
(pV qVr) A (=p V qV `r) true. Hence, it is unsatisfiable. < 


1.3.6 Applications of Satisfiability 


Many problems, in diverse areas such as robotics, software testing, artificial intelligence plan- 
ning, computer-aided design, machine vision, integrated circuit design, scheduling, computer 
networking, and genetics, can be modeled in terms of propositional satisfiability. Although most 
applications are quite complex and beyond the scope of this book, we can illustrate how two 
puzzles can be modeled as satisfiability problems. 


The n-Queens Problem The n-queens problem asks for a placement of n queens on ann X n 
chessboard so that no queen can attack another queen. This means that no two queens can be 
placed in the same row, in the same column, or on the same diagonal. We display a solution 
to the eight-queens problem in Figure 1. (The eight-queens problem dates back to 1848 when 
it was proposed by Max Bezzel and was completely solved by Franz Nauck in 1850. We will 
return to the n-queens problem in Section 11.4.) 

To model the n-queens problem as a satisfiability problem, we introduce n“ variables, p(i, j) 
for i = 1,2, ...,n and j = 1, 2, ...,n. For a given placement of a queens on the chessboard, 
p(i, j) is true when there is a queen on the square in the ith row and jth column, and is false 
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FIGURE 1 


otherwise. Note that squares (i, j) and (i’, j’) are on the same diagonal if either i+ i’ = j +’ or 
i—i! = j — j'. In the chessboard in Figure 1, p(6, 2) and p(2, 1) are true, while p(3, 4) and p(5, 4) 
are false. 

For no two of the n queens to be in the same row, there must be one queen in each row. We 
can show that there is one queen in each row by verifying that every row contains at least one 
queen and that every row contains at most one queen. We first note that Vi , PG j) asserts that 
row i contains at least one queen, and 


n n 


Q, = N VeGD 


i=l j=l 


asserts that every row contains at least one queen. 

For every row to include at most one queen, it must be the case that p(i, j) and p(k, j) are 
not both true for integers j and k with 1 < j < k < n. Observe that =p(i, j) V ap(i, k) asserts that 
at least one of p(i, j) and —p(i, k) is true, which means that at least one of p(i, j) and p(i, k) is 
false. So, to check that there is at most one queen in each row, we assert 


n n-i n 


o = AA A GD y pk j). 


i=l j=l k=j+1 
To assert that no column contains more than one queen, we assert that 


n n-l n 


Q = AA A GD y Pep). 


j=l i=l k=i+1 


(This assertion, together with the previous assertion that every row contains a queen, implies 
that every column contains a queen.) 
To assert that no diagonal contains two queens, we assert 


n n-l min(i-1,n-/) 


G=NA A @6Dyv i-k k+) 


i=2 j=l k=1 
and 


n—1 n—1 min(n—i,n—j) 


O=N\K A Cr idV-pG+ki+). 


i=l j=l k=l 


EXAMPLE 11 
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The innermost conjunction in Q, and in Q; for a pair (i, j) runs through the positions on a 
diagonal that begin at (i, j) and runs rightward along this diagonal. The upper limits on these 
innermost conjunctions identify the last cell in the board on each diagonal. 


Putting all this together, we find that the solutions of the n-queens problem are given 
by the assignments of truth values to the variables p(i,j), i= 1,2,...,n and j = 1,2,...,n 
that make 


Q=0,AQ,AQ03;AQ0,AQ5 


true. 

Using this and other approaches, the number of ways n queens can be placed on a chessboard 
so that no queen can attack another has been computed for n < 27. When n = 8 there are 92 
such placements, while for n = 16 this number grows to 14,772,512. (See the OEIS discussed 
in Section 2.4 for details.) < 


Sudoku Sudoku puzzles are constructed using a 9 x 9 grid made up of nine 3 x 3 subgrids, 
known as blocks, as shown in Figure 2. For each puzzle, some of the 81 cells, called givens, 
are assigned one of the numbers 1, 2, ..., 9, and the other cells are blank. The puzzle is solved 
by assigning a number to each blank cell so that every row, every column, and every one of the 
nine 3 x 3 blocks contains each of the nine possible numbers. Note that instead of using a 9 x 9 
grid, Sudoku puzzles can be based on n? x n? grids, for any positive integer n, with the n? x n? 
grid made up of n? n X n subgrids. 

The popularity of Sudoku dates back to the 1980s when it was introduced in Japan. It 
took 20 years for Sudoku to spread to rest of the world, but by 2005, Sudoku puzzles were a 
worldwide craze. The name Sudoku is short for the Japanese suuji wa dokushin ni kagiru, which 
means “the digits must remain single.” The modern game of Sudoku was apparently designed 
in the late 1970s by an American puzzle designer. The basic ideas of Sudoku date back even 
further; puzzles printed in French newspapers in the 1890s were quite similar, but not identical, 
to modern Sudoku. 

Sudoku puzzles designed for entertainment have two additional important properties. First, 
they have exactly one solution. Second, they can be solved using reasoning alone, that is, without 
resorting to searching all possible assignments of numbers to the cells. As a Sudoku puzzle is 
solved, entries in blank cells are successively determined by already known values. For instance, 
in the grid in Figure 2, the number 4 must appear in exactly one cell in the second row. How 
can we determine in which of the seven blank cells it must appear? First, we observe that 4 
cannot appear in one of the first three cells or in one of the last three cells of this row, because 
it already appears in another cell in the block each of these cells is in. We can also see that 4 


FIGURE2 A 9 x 9 Sudoku puzzle. 
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? 


It is tricky setting up the 
two inner indices so that 
all nine cells in each 
square block are 
examined. 


cannot appear in the fifth cell in this row, as it already appears in the fifth column in the fourth 
row. This means that 4 must appear in the sixth cell of the second row. 

Many strategies based on logic and mathematics have been devised for solving Sudoku 
puzzles (see [Dal0], for example). Here, we discuss one of the ways that have been devel- 
oped for solving Sudoku puzzles with the aid of a computer, which depends on modeling 
the puzzle as a propositional satisfiability problem. Using the model we describe, particular 
Sudoku puzzles can be solved using software developed to solve satisfiability problems. Cur- 
rently, Sudoku puzzles can be solved in less than 10 milliseconds this way. It should be noted 
that there are many other approaches for solving Sudoku puzzles via computers using other 
techniques. 

To encode a Sudoku puzzle, let p(i, j, n) denote the proposition that is true when the number 
n is in the cell in the ith row and jth column. There are 9 x 9 x 9 = 729 such propositions, as i, 
j, and n all range from 1 to 9. For example, for the puzzle in Figure 2, the number 6 is given as 
the value in the fifth row and first column. Hence, we see that p(5, 1, 6) is true, but p(5, j, 6) is 
false for j = 2,3, ..., 9. 

Given a particular Sudoku puzzle, we begin by encoding each of the given values. Then, 
we construct compound propositions that assert that every row contains every number, every 
column contains every number, every 3 X 3 block contains every number, and each cell contains 
no more than one number. It follows, as the reader should verify, that the Sudoku puzzle is solved 
by finding an assignment of truth values to the 729 propositions p(i, j, n) with i, j, and n each 
ranging from 1 to 9 that makes the conjunction of all these compound propositions true. After 
listing these assertions, we will explain how to construct the assertion that every row contains 
every integer from 1 to 9. We will leave the construction of the other assertions that every 
column contains every number and each of the nine 3 x 3 blocks contains every number to the 
exercises. 


> For each cell with a given value, we assert p(i, j, n) when the cell in row 7 and column j 
has the given value n. 


> We assert that every row contains every number: 


INA 


il nel j 


pij, n) 


9 
=1 


> We assert that every column contains every number: 
9 9 9 
ANA Veain 
> We assert that each of the nine 3 x 3 blocks contains every number: 


AAAV 


r=0 s=0 n=1 i=1 j=1 


2 2 9 3 3 
p3r+i3s+j,n) 


> To assert that no cell contains more than one number, we take the conjunction over all 
values of n, n’, i, and j, where each variable ranges from 1 to 9 and n # n' of p(i j, n) > 


pli, j, n’). 

We now explain how to construct the assertion that every row contains every number. 
First, to assert that row i contains the number n, we form Va p(ij,n). To assert that 
row i contains all n numbers, we form the conjunction of these disjunctions over all nine 
possible values of n, giving us A? Va p(i, j, n). Finally, to assert that every row contains 
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every number, we take the conjunction of Na Vi p(i, j, n) over all nine rows. This gives 


us Ni Na Vai p(i, j, n). (Exercises 71 and 72 ask for explanations of the assertions 
that every column contains every number and that each of the nine 3 x3 blocks contains 
every number.) 

Given a particular Sudoku puzzle, to solve this puzzle we can find a solution to the satisfi- 
ability problems that asks for a set of truth values for the 729 variables p(i, j, n) that makes the 
conjunction of all the listed assertions true. < 


1.3.7 Solving Satisfiability Problems 


A truth table can be used to determine whether a compound proposition is satisfiable, or 
equivalently, whether its negation is a tautology (see Exercise 64). This can be done by 
hand for a compound proposition with a small number of variables, but when the number 
of variables grows, this becomes impractical. For instance, there are 220 = 1,048,576 rows 
in the truth table for a compound proposition with 20 variables. Thus, you need a com- 
puter to help you determine, in this way, whether a compound proposition in 20 variables is 
satisfiable. 

When many applications are modeled, questions concerning the satisfiability of compound 
propositions with hundreds, thousands, or millions of variables arise. Note, for example, that 
when there are 1000 variables, checking every one of the 2!° (a number with more than 300 
decimal digits) possible combinations of truth values of the variables in a compound proposi- 
tion cannot be done by a computer in even trillions of years. No procedure is known that a com- 
puter can follow to determine in a reasonable amount of time whether an arbitrary compound 
proposition in such a large number of variables is satisfiable. However, progress has been made 
developing methods for solving the satisfiability problem for the particular types of compound 
propositions that arise in practical applications, such as for the solution of Sudoku puzzles. 
Many computer programs have been developed for solving satisfiability problems which have 
practical use. In our discussion of the subject of algorithms in Chapter 3, we will discuss this 
question further. In particular, we will explain the important role the propositional satisfiability 
problem plays in the study of the complexity of algorithms. 


HENRY MAURICE SHEFFER (1883-1964) Henry Maurice Sheffer, born to Jewish parents in the western 
Ukraine, emigrated to the United States in 1892 with his parents and six siblings. He studied at the Boston Latin 
School before entering Harvard, where he completed his undergraduate degree in 1905, his master’s in 1907, 
and his Ph.D. in philosophy in 1908. After holding a postdoctoral position at Harvard, Henry traveled to Europe 
on a fellowship. Upon returning to the United States, he became an academic nomad, spending one year each 
at the University of Washington, Cornell, the University of Minnesota, the University of Missouri, and City 
College in New York. In 1916 he returned to Harvard as a faculty member in the philosophy department. He 


| remained at Harvard until his retirement in 1952. 


Sheffer introduced what is now known as the Sheffer stroke in 1913; it became well known only after 
its use in the 1925 edition of Whitehead and Russell’s Principia Mathematica. In this same edition Russell 
wrote that Sheffer had invented a powerful method that could be used to simplify the Principia. Because of 
this comment, Sheffer was something of a mystery man to logicians, especially because Sheffer, who published 


little in his career, never published the details of this method, only describing it in mimeographed notes and in a brief published 


abstract. 


Sheffer was a dedicated teacher of mathematical logic. He liked his classes to be small and did not like auditors. When strangers 
appeared in his classroom, Sheffer would order them to leave, even his colleagues or distinguished guests visiting Harvard. Sheffer 
was barely five feet tall; he was noted for his wit and vigor, as well as for his nervousness and irritability. Although widely liked, he 
was quite lonely. He is noted for a quip he spoke at his retirement: “Old professors never die, they just become emeriti.” Sheffer is 
also credited with coining the term “Boolean algebra” (the subject of Chapter 12 of this text). Sheffer was briefly married and lived 
most of his later life in small rooms at a hotel packed with his logic books and vast files of slips of paper he used to jot down his 
ideas. Unfortunately, Sheffer suffered from severe depression during the last two decades of his life. 
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Exercises 


1. 


5. 


10. 


ES 11. 


12. 


Use truth tables to verify these equivalences. 


a) pAT=p b) pVF=p 
c) pAF=F d) pVvT=T 
e) pPVp=p f) pAp=p 


. Show that ~(=7p) and p are logically equivalent. 


Use truth tables to verify the commutative laws 


a) DVG=4qVp. b) PAG=qAp. 
Use truth tables to verify the associative laws 
a) (PVqQ)Vr=pV(qVr). 

b) PAQAr=PACGADN). 

P 


Use a truth table to verify the distributive law 
A(qVr)=(PAQ)V(PAP). 

Use a truth table to verify the first De Morgan law 
“(Pp Ag) = pv 74. 

U 


se De Morgan’s laws to find the negation of each of the 
following statements. 

a) Jan is rich and happy. 

b) Carlos will bicycle or run tomorrow. 

c) Mei walks or takes the bus to class. 

d) Ibrahim is smart and hard working. 


. Use De Morgan’s laws to find the negation of each of the 


following statements. 


a) Kwame will take a job in industry or go to graduate 
school. 

b) Yoshiko knows Java and calculus. 

c) James is young and strong. 

d) Rita will move to Oregon or Washington. 


. For each of these compound propositions, use the 


conditional-disjunction equivalence (Example 3) to find 
an equivalent compound proposition that does not in- 
volve conditionals. 


a) p> =q 

b) p >q)>r 

0) 0q >p) > P > ~q) 

For each of these compound propositions, use the 
conditional-disjunction equivalence (Example 3) to find 
an equivalent compound proposition that does not in- 
volve conditionals. 

a) =p > =q 

b) Vq) >p 

ce) p>) > p> q) 

Show that each of these conditional statements is a tau- 
tology by using truth tables. 


a) (pPAQ >p b) p> (pVq) 
©) p > (p> q) d) p^q)> (p >q) 
e) p >q4)>p f) =p > 4) > =q 


Show that each of these conditional statements is a tau- 
tology by using truth tables. 

a) prey] >q 

b) [p >49) >r] > p->r) 

©) [pPAP?-gl>4q 

d) [py pap > raq >r] >r 


13. Show that each conditional statement in Exercise 11 is 
a tautology using the fact that a conditional statement is 
false exactly when the hypothesis is true and the conclu- 
sion is false. (Do not use truth tables.) 


14. Show that each conditional statement in Exercise 12 is 
a tautology using the fact that a conditional statement is 
false exactly when the hypothesis is true and the conclu- 
sion is false. (Do not use truth tables.) 


15. Show that each conditional statement in Exercise 11 is a 
tautology by applying a chain of logical identities as in 
Example 8. (Do not use truth tables.) 


16. Show that each conditional statement in Exercise 12 is a 
tautology by applying a chain of logical identities as in 
Example 8. (Do not use truth tables.) 


17. Use truth tables to verify the absorption laws. 

a) PV(PAQ =P b) PAPVQ =P 
18. Determine whether (=p A (p > q)) > ~q is a tautology. 
19. Determine whether (7q A (p > q)) > =p is a tautology. 


Each of Exercises 20-32 asks you to show that two compound 
propositions are logically equivalent. To do this, either show 
that both sides are true, or that both sides are false, for ex- 
actly the same combinations of truth values of the proposi- 
tional variables in these expressions (whichever is easier). 


20. Show that p + q and (p Aq) V (Ap A ~q) are logically 
equivalent. 


21. Show that =(p e q) and p © ~q are logically equivalent. 
22. Show that p > q and ~q > 7p are logically equivalent. 
23. Show that =p + q and p © ~q are logically equivalent. 
24. Show that =(p ® q) and p © q are logically equivalent. 
25. Show that =(p e q) and =p + q are logically equivalent. 


26. Show that (p > q) ^ (p > r) and p > (q A r) are logi- 
cally equivalent. 


27. Show that (p > r) A (q > r) and (p V q) > r are logi- 
cally equivalent. 


28. Show that (p > q) V (p > r) and p > (q V r) are logi- 
cally equivalent. 


29. Show that (p > r) V (q > r) and (p ^q) > r are logi- 
cally equivalent. 


30. Show that =p > (q > r) and q > (p V r) are logically 
equivalent. 


31. Show that p © q and (p > q) A (q > p) are logically 
equivalent. 


32. Show that p + q and ap «+ ~q are logically equivalent. 
33. Show that (p > q) A^ (q > r) > (p > r) is a tautology. 


"34. Show that (p V q) A p V r) > (q V r) is a tautology. 


35. Show that (p > q) > r and p > (q > r) are not logically 
equivalent. 

36. Show that (p A q) > r and (p > r) A (q > r) are not log- 
ically equivalent. 


37. 


Show that (p>q)>(r>s) and 


(q > s) are not logically equivalent. 


(porno 


The dual of a compound proposition that contains only the 
logical operators V, A, and ~ is the compound proposition 
obtained by replacing each V by A, each A by V, each T 
by F, and each F by T. The dual of s is denoted by s*. 


38. 


39. 


40. 
41. 
42. 


#43, 


44. 


45. 


Find the dual of each of these compound propositions. 
a) pV =q b) p^ (4y CAT) 

©) (p^) V (qF) 

Find the dual of each of these compound propositions. 
a) pA7qA7r b) PAGAN)Vs 

© PVE)AGVT) 

When does s* = s, where s is a compound proposition? 
Show that (s*)* = s when s is a compound proposition. 
Show that the logical equivalences in Table 6, except for 
the double negation law, come in pairs, where each pair 
contains compound propositions that are duals of each 
other. 

Why are the duals of two equivalent compound propo- 
sitions also equivalent, where these compound proposi- 
tions contain only the operators ^, V, and =? 

Find a compound proposition involving the propositional 
variables p, q, and r that is true when p and q are true 
and r is false, but is false otherwise. [Hint: Use a con- 
junction of each propositional variable or its negation.] 
Find a compound proposition involving the propositional 
variables p, q, and r that is true when exactly two of p, q, 
and r are true and is false otherwise. [Hint: Form a dis- 
junction of conjunctions. Include a conjunction for each 
combination of values for which the compound proposi- 
tion is true. Each conjunction should include each of the 
three propositional variables or its negations. ] 


. Suppose that a truth table in n propositional variables is 


specified. Show that a compound proposition with this 
truth table can be formed by taking the disjunction of 
conjunctions of the variables or their negations, with one 
conjunction included for each combination of values for 
which the compound proposition is true. The resulting 
compound proposition is said to be in disjunctive nor- 
mal form. 


A collection of logical operators is called functionally com- 
plete if every compound proposition is logically equivalent 
to a compound proposition involving only these logical oper- 
ators. 


47. 


* 48, 


* 49, 


Show that =, A, and V form a functionally complete col- 
lection of logical operators. [Hint: Use the fact that ev- 
ery compound proposition is logically equivalent to one 
in disjunctive normal form, as shown in Exercise 46.] 
Show that ~ and A form a functionally complete col- 
lection of logical operators. [Hint: First use a De Mor- 
gan law to show that p Vq is logically equivalent to 
aap A7q).] 

Show that ~ and v form a functionally complete collec- 
tion of logical operators. 


We now present a group of exercises that involve the logical 
operators NAND and NOR. The proposition p NAND {q is true 
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when either p or q, or both, are false; and it is false when both p 
and q are true. The proposition p NOR q is true when both 
p and q are false, and it is false otherwise. The propositions 
p NAND q and p NOR q are denoted by p | gandp | q, respec- 
tively. (The operators | and | are called the Sheffer stroke and 
the Peirce arrow after H. M. Sheffer and C. S. Peirce, respec- 
tively.) 


50. 
51. 
52. 
53. 
54. 


#55. 


56. 


57. 


58. 


* 59. 


60. 


61. 


62. 


63. 


64. 


65. 


Construct a truth table for the logical operator NAND. 
Show that p | q is logically equivalent to =(p A q). 
Construct a truth table for the logical operator NOR. 
Show that p | q is logically equivalent to =(p V q). 

In this exercise we will show that {|} is a functionally 

complete collection of logical operators. 

a) Show that p | p is logically equivalent to =p. 

b) Show that (p | q) | (p 4 q) is logically equivalent to 
PYG. 

c) Conclude from parts (a) and (b), and Exercise 49, that 
{J} is a functionally complete collection of logical 
operators. 

Find a compound proposition logically equivalent to 
p > q using only the logical operator |. 
Show that {|} is a functionally complete collection of log- 
ical operators. 
Show that p | q and q | p are equivalent. 
Show that p | (q | r) and (p | q) | r are not equivalent, so 
that the logical operator | is not associative. 
How many different truth tables of compound proposi- 
tions are there that involve the propositional variables p 
and q? 
Show that if p, g, and r are compound propositions such 
that p and q are logically equivalent and q and r are log- 
ically equivalent, then p and r are logically equivalent. 
The following sentence is taken from the specification of 
a telephone system: “If the directory database is opened, 
then the monitor is put in a closed state, if the sys- 
tem is not in its initial state.” This specification is hard 
to understand because it involves two conditional state- 
ments. Find an equivalent, easier-to-understand specifi- 
cation that involves disjunctions and negations but not 
conditional statements. 

How many of the disjunctions p V =q, ap Vq, qgVr, 

qV ^r, and ~q V =r can be made simultaneously true by 

an assignment of truth values to p, g, and r? 

How many of the disjunctions p V ^q V s, =p V ar V S, 

apy ary as, apVqgVrs, 4 Vros, qV rV sS, 

ap Vaq Vas, pVrV s, and pVr Vans can be made si- 

multaneously true by an assignment of truth values to p, 

q, r, and s? 

Show that the negation of an unsatisfiable compound 

proposition is a tautology and the negation of a com- 

pound proposition that is a tautology is unsatisfiable. 

Determine whether each of these compound propositions 

is satisfiable. 

a) (p Vag) A (PV q) A (PV 79) 

b) p > DAR > DACP > DACP > 79) 

ce) PegQrAcer®” 
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66. 


67. 


68. 
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Determine whether each of these compound propositions 
is satisfiable. 
a) PV gV-7nA(DV7qGV 75) A (pV ArV as) A 
(=p V 7g V 75) A (pV gV 78) 
b) (GpV7qVr)A (PV qGV78)A(—~V7AqgV 75) A 
(=p V-7ArVvas)A (pV qGV 7r)A (pV 7rVvn7s) 
c) PVgGVTA(PV7qgV 78) A(GV 7rV s) Apv 
rVs)A(ApVqV7s)A (pV 7qgV ar) Apy 
aq V 8) A (Ap V rV ns) 
Find the compound proposition Q constructed in Exam- 
ple 10 for the n-queens problem, and use it to find all the 
ways that n queens can be placed on an n X n chessboard, 
so that no queen can attack another when n is 
a) 2. b) 3. c) 4. 
Starting with the compound proposition Q found in Ex- 
ample 10, construct a compound proposition that can be 


69. 


70. 


71. 


* 72. 


used to find all solutions of the n-queens problem where 
the queen in the first column is in an odd-numbered row. 


Show how the solution of a given 4 x 4 Sudoku puzzle 
can be found by solving a satisfiability problem. 


Construct a compound proposition that asserts that ev- 
ery cell of a 9x9 Sudoku puzzle contains at least one 
number. 


Explain the steps in the construction of the com- 
pound proposition given in the text that asserts that 
every column of a 9X9 Sudoku puzzle contains 
every number. 


Explain the steps in the construction of the compound 
proposition given in the text that asserts that each of 
the nine 3 x 3 blocks of a 9 x 9 Sudoku puzzle contains 
every number. 


1.4.1 Introduction 


Propositional logic, studied in Sections 1.1—1.3, cannot adequately express the meaning of all 
statements in mathematics and in natural language. For example, suppose that we know that 


“Every computer connected to the university network is functioning properly.” 


No rules of propositional logic allow us to conclude the truth of the statement 


“MATHS is functioning properly,” 


where MATHS is one of the computers connected to the university network. Likewise, we can- 
not use the rules of propositional logic to conclude from the statement 


“CS2 is under attack by an intruder,” 


where CS2 is a computer on the university network, to conclude the truth of 


“There is a computer on the university network that is under attack by an intruder.” 


In this section we will introduce a more powerful type of logic called predicate logic. We 
will see how predicate logic can be used to express the meaning of a wide range of statements 
in mathematics and computer science in ways that permit us to reason and explore relationships 
between objects. To understand predicate logic, we first need to introduce the concept of a 
predicate. Afterward, we will introduce the notion of quantifiers, which enable us to reason 
with statements that assert that a certain property holds for all objects of a certain type and with 
statements that assert the existence of an object with a particular property. 


1.4.2 Predicates 


Statements involving variables, such as 


sy > 3 “ey = y + 3” 


sy +y = z 


EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


Extra » 
Examples 


EXAMPLE 4 
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and 

“Computer x is under attack by an intruder,” 
and 

“Computer x is functioning properly,” 


are often found in mathematical assertions, in computer programs, and in system specifications. 
These statements are neither true nor false when the values of the variables are not specified. In 
this section, we will discuss the ways that propositions can be produced from such statements. 

The statement “x is greater than 3” has two parts. The first part, the variable x, is the subject 
of the statement. The second part—the predicate, “is greater than 3”—+efers to a property that 
the subject of the statement can have. We can denote the statement “x is greater than 3” by P(x), 
where P denotes the predicate “is greater than 3” and x is the variable. The statement P(x) is also 
said to be the value of the propositional function P at x. Once a value has been assigned to the 
variable x, the statement P(x) becomes a proposition and has a truth value. Consider Examples 1 
and 2. 


Let P(x) denote the statement “x > 3.” What are the truth values of P(4) and P(2)? 


Solution: We obtain the statement P(4) by setting x = 4 in the statement “x > 3.” Hence, 
P(4), which is the statement “4 > 3,” is true. However, P(2), which is the statement “2 > 3,” 
is false. < 


Let A(x) denote the statement “Computer x is under attack by an intruder.” Suppose that of the 
computers on campus, only CS2 and MATH are currently under attack by intruders. What are 
truth values of A(CS1), A(CS2), and ACMATH1)? 


Solution: We obtain the statement A(CS1) by setting x = CS1 in the statement “Computer x 
is under attack by an intruder.” Because CS1 is not on the list of computers currently under 
attack, we conclude that A(CS1) is false. Similarly, because CS2 and MATH] are on the list of 
computers under attack, we know that A(CS2) and A(MATH1) are true. < 


We can also have statements that involve more than one variable. For instance, consider the 
statement “x = y + 3.” We can denote this statement by Q(x, y), where x and y are variables and 
Q is the predicate. When values are assigned to the variables x and y, the statement Q(x, y) has 
a truth value. 


Let Q(x, y) denote the statement “x = y + 3.” What are the truth values of the propositions 
Q(1, 2) and Q(3, 0)? 


Solution: To obtain Q(1, 2), set x = 1 and y = 2 in the statement Q(x, y). Hence, Q(1, 2) is the 
statement “1 = 2 +3,’ which is false. The statement Q(3, 0) is the proposition “3 = 0 + 3,” 
which is true. < 


Let A(c, n) denote the statement “Computer c is connected to network n,” where c is a variable 
representing a computer and n is a variable representing a network. Suppose that the computer 
MATH I is connected to network CAMPUS2, but not to network CAMPUS1. What are the 
values of AMATH1, CAMPUS1) and A(MATH1, CAMPUS2)? 
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Solution: Because MATH is not connected to the CAMPUS1 network, we see that AAMATHI, 
CAMPUS1) is false. However, because MATH is connected to the CAMPUS2 network, we 
see that AMATHI, CAMPUS2) is true. < 


Similarly, we can let R(x, y, z) denote the statement “x + y = z.” When values are assigned 
to the variables x, y, and z, this statement has a truth value. 


EXAMPLE 5 What are the truth values of the propositions R(1, 2, 3) and R(O, 0, 1)? 


Solution: The proposition R(1, 2, 3) is obtained by setting x = 1, y = 2, and z = 3 in the state- 
ment R(x, y, z). We see that R(1, 2, 3) is the statement “1 + 2 = 3,” which is true. Also note that 


R(O, 0, 1), which is the statement “0 + 0 = 1,” is false. < 
In general, a statement involving the n variables x,, xy, ..., x,, can be denoted by 
P(X1,Xq, »., Xp): 
A statement of the form P(x,, x, ...,x,,) is the value of the propositional function P at the 
n-tuple (x,, X2, ...,x,,), and P is also called an n-place predicate or an n-ary predicate. 


Propositional functions occur in computer programs, as Example 6 demonstrates. 


EXAMPLE 6 Consider the statement 


if x > Othenx:=x+1. 


Links » 


CHARLES SANDERS PEIRCE (1839-1914) Many consider Charles Peirce, born in Cambridge, Mas- 
sachusetts, to be the most original and versatile American intellect. He made important contributions to an 
amazing number of disciplines, including mathematics, astronomy, chemistry, geodesy, metrology, engineer- 
ing, psychology, philology, the history of science, and economics. Peirce was also an inventor, a lifelong student 
of medicine, a book reviewer, a dramatist and an actor, a short story writer, a phenomenologist, a logician, and 
a metaphysician. He is noted as the preeminent system-building philosopher competent and productive in logic, 
mathematics, and a wide range of sciences. He was encouraged by his father, Benjamin Peirce, a professor of 
mathematics and natural philosophy at Harvard, to pursue a career in science. Instead, he decided to study logic 
and scientific methodology. Peirce attended Harvard (1855-1859) and received a Harvard master of arts degree 
(1862) and an advanced degree in chemistry from the Lawrence Scientific School (1863). 

In 1861, Peirce became an aide in the U.S. Coast Survey, with the goal of better understanding scientific methodology. His 
service for the Survey exempted him from military service during the Civil War. While working for the Survey, Peirce did astro- 
nomical and geodesic work. He made fundamental contributions to the design of pendulums and to map projections, applying new 
mathematical developments in the theory of elliptic functions. He was the first person to use the wavelength of light as a unit of 
measurement. Peirce rose to the position of Assistant for the Survey, a position he held until forced to resign in 1891 when he 
disagreed with the direction taken by the Survey’s new administration. 

While making his living from work in the physical sciences, Peirce developed a hierarchy of sciences, with mathematics at the 
top rung, in which the methods of one science could be adapted for use by those sciences under it in the hierarchy. During this time, 
he also founded the American philosophical theory of pragmatism. 

The only academic position Peirce ever held was lecturer in logic at Johns Hopkins University in Baltimore (1879-1884). His 
mathematical work during this time included contributions to logic, set theory, abstract algebra, and the philosophy of mathematics. 
His work is still relevant today, with recent applications to artificial intelligence. Peirce believed that the study of mathematics could 
develop the mind’s powers of imagination, abstraction, and generalization. His diverse activities after retiring from the Survey 
included writing for periodicals, contributing to scholarly dictionaries, translating scientific papers, guest lecturing, and textbook 
writing. Unfortunately, his income from these pursuits was insufficient to protect him and his second wife from abject poverty. He 
was supported in his later years by a fund created by his many admirers and administered by the philosopher William James, his 
lifelong friend. Although Peirce wrote and published voluminously in a vast range of subjects, he left more than 100,000 pages 
of unpublished manuscripts. Because of the difficulty of studying his unpublished writings, scholars have only recently started to 
understand some of his varied contributions. A group of people is devoted to making his work available over the Internet to bring a 
better appreciation of Peirce’s accomplishments to the world. 
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When this statement is encountered in a program, the value of the variable x at that point in the 
execution of the program is inserted into P(x), which is “x > 0.” If P(x) is true for this value 
of x, the assignment statement x := x + 1 is executed, so the value of x is increased by 1. If 
P(x) is false for this value of x, the assignment statement is not executed, so the value of x is 
not changed. < 


PRECONDITIONS AND POSTCONDITIONS Predicates are also used to establish the 
correctness of computer programs, that is, to show that computer programs always produce 
the desired output when given valid input. (Note that unless the correctness of a computer 
program is established, no amount of testing can show that it produces the desired output 
for all input values, unless every input value is tested.) The statements that describe valid 
input are known as preconditions and the conditions that the output should satisfy when the 
program has run are known as postconditions. As Example 7 illustrates, we use predicates to 
describe both preconditions and postconditions. We will study this process in greater detail in 
Section 5.5. 


Consider the following program, designed to interchange the values of two variables x and y. 


temp := X 
X i= y 
y := temp 


Find predicates that we can use as the precondition and the postcondition to verify the correct- 
ness of this program. Then explain how to use them to verify that for all valid input the program 
does what is intended. 


Solution: For the precondition, we need to express that x and y have particular values before we 
run the program. So, for this precondition we can use the predicate P(x, y), where P(x, y) is the 
statement “x = a and y = b,” where a and b are the values of x and y before we run the program. 
Because we want to verify that the program swaps the values of x and y for all input values, for 
the postcondition we can use Q(x, y), where Q(x, y) is the statement “x = b and y = a.” 

To verify that the program always does what it is supposed to do, suppose that the pre- 
condition P(x, y) holds. That is, we suppose that the statement “x = a and y = b” is true. This 
means that x = a and y = b. The first step of the program, temp := x, assigns the value of x to 
the variable temp, so after this step we know that x = a, temp = a, and y = b. After the second 
step of the program, x := y, we know that x = b, temp = a, and y = b. Finally, after the third 
step, we know that x = b, temp = a, and y = a. Consequently, after this program is run, the 
postcondition Q(x, y) holds, that is, the statement “x = b and y = a” is true. < 


1.4.3 Quantifiers 


When the variables in a propositional function are assigned values, the resulting statement be- 
comes a proposition with a certain truth value. However, there is another important way, called 
quantification, to create a proposition from a propositional function. Quantification expresses 
the extent to which a predicate is true over a range of elements. In English, the words all, some, 
many, none, and few are used in quantifications. We will focus on two types of quantification 
here: universal quantification, which tells us that a predicate is true for every element under 
consideration, and existential quantification, which tells us that there is one or more element 
under consideration for which the predicate is true. The area of logic that deals with predicates 
and quantifiers is called the predicate calculus. 
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Remember that the truth 
value of YxP(x) depends 
on the domain! 


THE UNIVERSAL QUANTIFIER Many mathematical statements assert that a property is 
true for all values of a variable in a particular domain, called the domain of discourse (or the 
universe of discourse), often just referred to as the domain. Such a statement is expressed us- 
ing universal quantification. The universal quantification of P(x) for a particular domain is the 
proposition that asserts that P(x) is true for all values of x in this domain. Note that the domain 
specifies the possible values of the variable x. The meaning of the universal quantification of 
P(x) changes when we change the domain. The domain must always be specified when a uni- 
versal quantifier is used; without it, the universal quantification of a statement is not defined. 


The universal quantification of P(x) is the statement 
“P(x) for all values of x in the domain.” 


The notation VxP(x) denotes the universal quantification of P(x). Here V is called the 
universal quantifier. We read VxP(x) as “for all xP(x)” or “for every xP(x).” An element for 
which P(x) is false is called a counterexample to VxP(.). 


The meaning of the universal quantifier is summarized in the first row of Table 1. We illus- 
trate the use of the universal quantifier in Examples 8-12 and 15. 


Let P(x) be the statement “x + 1 > x.” What is the truth value of the quantification VxP(x), where 
the domain consists of all real numbers? 


Solution: Because P(x) is true for all real numbers x, the quantification 
VxP(x) 
is true. < 


Remark: Generally, an implicit assumption is made that all domains of discourse for quantifiers 
are nonempty. Note that if the domain is empty, then VxP(x) is true for any propositional function 
P(x) because there are no elements x in the domain for which P(x) is false. 


Besides “for all” and “for every,” universal quantification can be expressed in many other 
ways, including “all of,” “for each,” “given any,” “for arbitrary,” “for each,” and “for any.” 


Remark: It is best to avoid using “for any x” because it is often ambiguous as to whether “any” 
means “every” or “some.” In some cases, “any” is unambiguous, such as when it is used in 
negatives: “There is not any reason to avoid studying.” 


A statement VxP(x) is false, where P(x) is a propositional function, if and only if P(x) is not 
always true when x is in the domain. One way to show that P(x) is not always true when x is in the 
domain is to find a counterexample to the statement VxP(x). Note that a single counterexample is 
all we need to establish that VxP(x) is false. Example 9 illustrates how counterexamples are used. 


TABLE 1 Quantifiers. 


VxP(x) P(x) is true for every x. There is an x for which P(x) is false. 
AxP(x) There is an x for which P(x) is true. P(x) is false for every x. 
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Let Q(x) be the statement “x < 2.” What is the truth value of the quantification VxQ(x), where 
the domain consists of all real numbers? 


Solution: Q(x) is not true for every real number x, because, for instance, Q(3) is false. That is, 
x = 3 is acounterexample for the statement VxQ(x). Thus, 


VxQ(x) 
is false. < 


Suppose that P(x) is “x4 > 0.” To show that the statement VxP(x) is false where the universe of 
discourse consists of all integers, we give a counterexample. We see that x = 0 is a counterex- 
ample because x? = 0 when x = 0, so that x? is not greater than 0 when x = 0. < 


Looking for counterexamples to universally quantified statements is an important activity 
in the study of mathematics, as we will see in subsequent sections of this book. 


What does the statement YxN (x) mean if N(x) is “Computer x is connected to the network” and 
the domain consists of all computers on campus? 


Solution: The statement VxN(x) means that for every computer x on campus, that computer x is 
connected to the network. This statement can be expressed in English as “Every computer on 
campus is connected to the network.” < 


As we have pointed out, specifying the domain is mandatory when quantifiers are used. The 
truth value of a quantified statement often depends on which elements are in this domain, as 
Example 12 shows. 


What is the truth value of Vx(x? > x) if the domain consists of all real numbers? What is the 
truth value of this statement if the domain consists of all integers? 


Solution: The universal quantification Vx(x? > x), where the domain consists of all real num- 
bers, is false. For example, GY ž L. Note that x? > x if and only if x? — x = x(x — 1) > 0. Con- 
sequently, x > xif and only if x < 0 or x > 1. It follows that Yx(x? > x) is false if the domain 
consists of all real numbers (because the inequality is false for all real numbers x with 0 < x < 1). 
However, if the domain consists of the integers, Yx(x? > x) is true, because there are no integers 
x withO<x <1. < 


THE EXISTENTIAL QUANTIFIER Many mathematical statements assert that there is an 
element with a certain property. Such statements are expressed using existential quantification. 
With existential quantification, we form a proposition that is true if and only if P(x) is true for 
at least one value of x in the domain. 


The existential quantification of P(x) is the proposition 
“There exists an element x in the domain such that P(x).” 


We use the notation 4xP(x) for the existential quantification of P(x). Here J is called the 
existential quantifier. 
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Remember that the truth 
value of 4xP(x) depends 
on the domain! 


A domain must always be specified when a statement 4xP(x) is used. Furthermore, the 
meaning of 4xP(x) changes when the domain changes. Without specifying the domain, the state- 
ment 4xP(x) has no meaning. 

Besides the phrase “there exists,’ we can also express existential quantification in many 
other ways, such as by using the words “for some,” “for at least one,” or “there is.” The existential 
quantification 4xP(x) is read as 


“There is an x such that P(x),” 
“There is at least one x such that P(x),” 


or 
“For some xP(x).” 


The meaning of the existential quantifier is summarized in the second row of Table 1. We 
illustrate the use of the existential quantifier in Examples 13, 14, and 16. 


Let P(x) denote the statement “x > 3.” What is the truth value of the quantification 4xP(x), 
where the domain consists of all real numbers? 


Solution: Because “x > 3” is sometimes true—for instance, when x = 4—the existential quan- 
tification of P(x), which is 4xP(x), is true. < 


Observe that the statement 4xP(x) is false if and only if there is no element x in the domain 
for which P(x) is true. That is, 3xP(x) is false if and only if P(x) is false for every element of the 
domain. We illustrate this observation in Example 14. 


Let Q(x) denote the statement “x = x + 1.” What is the truth value of the quantification 4xQ(x), 
where the domain consists of all real numbers? 


Solution: Because Q(x) is false for every real number x, the existential quantification of Q(x), 
which is 4xQ(x), is false. < 


Remark: Generally, an implicit assumption is made that all domains of discourse for quantifiers 
are nonempty. If the domain is empty, then 4xQ(x) is false whenever Q(x) is a propositional 
function because when the domain is empty, there can be no element x in the domain for which 
Q(x) is true. 


THE UNIQUENESS QUANTIFIER We have now introduced universal and existential quan- 
tifiers. These are the most important quantifiers in mathematics and computer science. However, 
there is no limitation on the number of different quantifiers we can define, such as “there are 
exactly two,” “there are no more than three,” “there are at least 100,” and so on. Of these other 
quantifiers, the one that is most often seen is the uniqueness quantifier, denoted by 3! or 3). 
The notation 4!xP(x) [or 4,xP(x)] states “There exists a unique x such that P(x) is true.” (Other 
phrases for uniqueness quantification include “there is exactly one” and “there is one and only 
one.”) For instance, 4!x(x — 1 = 0), where the domain is the set of real numbers, states that 
there is a unique real number x such that x — 1 = 0. This is a true statement, as x = 1 is the 
unique real number such that x — 1 = 0. Observe that we can use quantifiers and propositional 
logic to express uniqueness (see Exercise 52 in Section 1.5), so the uniqueness quantifier can 
be avoided. Generally, it is best to stick with existential and universal quantifiers so that rules 
of inference for these quantifiers can be used. 


EXAMPLE 15 


EXAMPLE 16 


1.4 Predicates and Quantifiers 47 


1.4.4 QUANTIFIERS OVER FINITE DOMAINS 


When the domain of a quantifier is finite, that is, when all its elements can be listed, quantified 
statements can be expressed using propositional logic. In particular, when the elements of the 
domain are x), X2, ..., X,, where n is a positive integer, the universal quantification VxP(x) is the 
same as the conjunction 


P(x) A Pa) Avs A P), 
because this conjunction is true if and only if P(x,), P(x), ..., P(x,,) are all true. 


What is the truth value of VxP(x), where P(x) is the statement “x? < 10” and the domain consists 
of the positive integers not exceeding 4? 


Solution: The statement VxP(x) is the same as the conjunction 
P(1) A P(2) A P(S) A P(A), 


because the domain consists of the integers 1, 2, 3, and 4. Because P(4), which is the statement 
“4? < 10,” is false, it follows that VxP(x) is false. < 


Similarly, when the elements of the domain are x), X2, ..., x,, Where n is a positive integer, 
the existential quantification 4xP(x) is the same as the disjunction 


P(x) V P(X) V = V Pœ), 
because this disjunction is true if and only if at least one of P(x,), P@2), ..., P(x,,) is true. 


What is the truth value of 3xP(x), where P(x) is the statement “x? > 10” and the universe of 
discourse consists of the positive integers not exceeding 4? 


Solution: Because the domain is {1, 2, 3, 4}, the proposition 4xP(x) is the same as the disjunction 
P(1) v P(2) v PB) V P(A). 


Because P(4), which is the statement “42 > 10,” is true, it follows that 3xP(x) is true. < 


CONNECTIONS BETWEEN QUANTIFICATION AND LOOPING Itis sometimes helpful 
to think in terms of looping and searching when determining the truth value of a quantification. 
Suppose that there are n objects in the domain for the variable x. To determine whether VxP(x) 
is true, we can loop through all n values of x to see whether P(x) is always true. If we encounter 
a value x for which P(x) is false, then we have shown that VxP(x) is false. Otherwise, VxP(x) 
is true. To see whether 4xP(x) is true, we loop through the n values of x searching for a value 
for which P(x) is true. If we find one, then 4xP(x) is true. If we never find such an x, then we 
have determined that 4xP(x) is false. (Note that this searching procedure does not apply if there 
are infinitely many values in the domain. However, it is still a useful way of thinking about the 
truth values of quantifications.) 
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1.4.5 Quantifiers with Restricted Domains 


An abbreviated notation is often used to restrict the domain of a quantifier. In this nota- 
tion, a condition a variable must satisfy is included after the quantifier. This is illustrated in 
Example 17. We will also describe other forms of this notation involving set membership in 
Section 2.1. 


What do the statements Vx < 0 (x? > 0), Vy # 0 0? Æ 0), and Jz > 0 (z? = 2) mean, where the 
domain in each case consists of the real numbers? 


Solution: The statement Vx < 0 (x? > 0) states that for every real number x with x < 0, x? > 0. 
That is, it states “The square of a negative real number is positive.” This statement is the same 
as Yx(x < 0 > xX > 0). 

The statement Vy Æ 0 (y? # 0) states that for every real number y with y # 0, we have y? £ 
0. That is, it states “The cube of every nonzero real number is nonzero.” This statement is 
equivalent to Vy(y #0 > y? #0). 

Finally, the statement 3z > 0 (z? = 2) states that there exists a real number z with z > 0 such 
that z? = 2. That is, it states “There is a positive square root of 2.” This statement is equivalent 
to Izz > 0AZ =2). < 


Note that the restriction of a universal quantification is the same as the universal quantifi- 
cation of a conditional statement. For instance, Vx < 0 (x? > 0) is another way of expressing 
Yx(x < 0 > x? > 0). On the other hand, the restriction of an existential quantification is the 
same as the existential quantification of a conjunction. For instance, 3z > 0 (z? = 2) is another 
way of expressing 4z(z > 0 A 2? = 2). 


1.4.6 Precedence of Quantifiers 


The quantifiers V and 4 have higher precedence than all logical operators from propositional 
calculus. For example, VxP(x) V Q(x) is the disjunction of VxP(x) and Q(x). In other words, it 
means (VxP(x)) V Q(x) rather than Vx(P(x) V Q(x)). 


1.4.7 Binding Variables 


When a quantifier is used on the variable x, we say that this occurrence of the variable is bound. 
An occurrence of a variable that is not bound by a quantifier or set equal to a particular value 
is said to be free. All the variables that occur in a propositional function must be bound or set 
equal to a particular value to turn it into a proposition. This can be done using a combination of 
universal quantifiers, existential quantifiers, and value assignments. 

The part of a logical expression to which a quantifier is applied is called the scope of this 
quantifier. Consequently, a variable is free if it is outside the scope of all quantifiers in the 
formula that specify this variable. 


In the statement 4x(x + y = 1), the variable x is bound by the existential quantification 3x, but 
the variable y is free because it is not bound by a quantifier and no value is assigned to this 
variable. This illustrates that in the statement 4x(x + y = 1), x is bound, but y is free. 

In the statement 4x(P(x) A Q(x)) V YxR(x), all variables are bound. The scope of the first 
quantifier, 4x, is the expression P(x) A Q(x), because 4x is applied only to P(x) A Q(x) and not 
to the rest of the statement. Similarly, the scope of the second quantifier, Vx, is the expression 
R(x). That is, the existential quantifier binds the variable x in P(x) A Q(x) and the universal 
quantifier Vx binds the variable x in R(x). Observe that we could have written our statement 
using two different variables x and y, as 4x(P(x) A Q(x)) V VyR(y), because the scopes of the 
two quantifiers do not overlap. The reader should be aware that in common usage, the same 
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letter is often used to represent variables bound by different quantifiers with scopes that do not 
overlap. < 


1.4.8 Logical Equivalences Involving Quantifiers 


In Section 1.3 we introduced the notion of logical equivalences of compound propositions. We 
can extend this notion to expressions involving predicates and quantifiers. 


Statements involving predicates and quantifiers are logically equivalent if and only if they 
have the same truth value no matter which predicates are substituted into these statements 
and which domain of discourse is used for the variables in these propositional functions. 
We use the notation S = T to indicate that two statements S and T involving predicates and 
quantifiers are logically equivalent. 


Example 19 illustrates how to show that two statements involving predicates and quantifiers 
are logically equivalent. 


Show that Vx(P(x) A Q(x)) and VxP(x) A VxQ(x) are logically equivalent (where the same do- 
main is used throughout). This logical equivalence shows that we can distribute a universal 
quantifier over a conjunction. Furthermore, we can also distribute an existential quantifier over 
a disjunction. However, we cannot distribute a universal quantifier over a disjunction, nor can 
we distribute an existential quantifier over a conjunction. (See Exercises 52 and 53.) 


Solution: To show that these statements are logically equivalent, we must show that they always 
take the same truth value, no matter what the predicates P and Q are, and no matter which 
domain of discourse is used. Suppose we have particular predicates P and Q, with a common 
domain. We can show that Yx(P(x) A Q(x)) and VxP(x) A YxQ(x) are logically equivalent by 
doing two things. First, we show that if Vx(P(x) A Q(x)) is true, then VxP(x) A VxQ(x) is true. 
Second, we show that if VxP(x) A VxQ(x) is true, then Vx(P(x) A Q(x)) is true. 

So, suppose that Vx(P(x) A Q(x)) is true. This means that if a is in the domain, then P(a) A 
Q(a) is true. Hence, P(a) is true and Q(a) is true. Because P(a) is true and Q(a) is true for every 
element a in the domain, we can conclude that VxP(x) and VxQ(x) are both true. This means 
that VxP(x) A VxQ(x) is true. 

Next, suppose that VxP(x) A VxQ(x) is true. It follows that VxP(x) is true and VxQ(x) is 
true. Hence, if a is in the domain, then P(a) is true and Q(a) is true [because P(x) and Q(x) are 
both true for all elements in the domain, there is no conflict using the same value of a here]. 
It follows that for all a, P(a) A O(a) is true. It follows that Vx(P(x) A Q(x)) is true. We can now 
conclude that 


Vx(P(x) A O(x)) = VxP(x) A YxQ(x). < 


1.4.9 Negating Quantified Expressions 


We will often want to consider the negation of a quantified expression. For instance, consider 
the negation of the statement 


“Every student in your class has taken a course in calculus.” 
This statement is a universal quantification, namely, 


VxP(x), 
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where P(x) is the statement “x has taken a course in calculus” and the domain consists of the 
students in your class. The negation of this statement is “It is not the case that every student in 
your class has taken a course in calculus.” This is equivalent to “There is a student in your class 
who has not taken a course in calculus.” And this is simply the existential quantification of the 
negation of the original propositional function, namely, 


Ax P(x), 
This example illustrates the following logical equivalence: 
AVxP(x) = Ax AP(x). 


To show that =VxP(x) and 3xP(x) are logically equivalent no matter what the propositional 
function P(x) is and what the domain is, first note that ~VxP(x) is true if and only if VxP(x) is 
false. Next, note that VxP(x) is false if and only if there is an element x in the domain for which 
P(x) is false. This holds if and only if there is an element x in the domain for which =P(x) is 
true. Finally, note that there is an element x in the domain for which —P(x) is true if and only if 
dx +P(x) is true. Putting these steps together, we can conclude that =VxP(x) is true if and only 
if 4x =P(x) is true. It follows that ~VxP(x) and 4x P(x) are logically equivalent. 

Suppose we wish to negate an existential quantification. For instance, consider the proposi- 
tion “There is a student in this class who has taken a course in calculus.” This is the existential 
quantification 


AxQ(x), 


where Q(x) is the statement “x has taken a course in calculus.” The negation of this statement is 
the proposition “It is not the case that there is a student in this class who has taken a course in 
calculus.” This is equivalent to “Every student in this class has not taken calculus,” which is just 
the universal quantification of the negation of the original propositional function, or, phrased in 
the language of quantifiers, 


Vx 3Q(x). 


This example illustrates the equivalence 


73xO(x) = Vx 7Q(x). 


To show that ~3xQ(x) and Vx =Q(x) are logically equivalent no matter what Q(x) is and what 
the domain is, first note that ~AxQ(x) is true if and only if 3xQ(x) is false. This is true if and 
only if no x exists in the domain for which Q(x) is true. Next, note that no x exists in the domain 
for which Q(x) is true if and only if Q(x) is false for every x in the domain. Finally, note that 
Q(x) is false for every x in the domain if and only if =Q(x) is true for all x in the domain, which 
holds if and only if Vx7Q(x) is true. Putting these steps together, we see that =4xQ(x) is true if 
and only if Vx7Q(x) is true. We conclude that ~3xQ(x) and Vx =Q(x) are logically equivalent. 

The rules for negations for quantifiers are called De Morgan’s laws for quantifiers. These 
rules are summarized in Table 2. 


Remark: When the domain of a predicate P(x) consists of n elements, where n is a positive 
integer greater than one, the rules for negating quantified statements are exactly the same as De 
Morgan’s laws discussed in Section 1.3. This is why these rules are called De Morgan’s laws for 
quantifiers. When the domain has n elements x,, x5, ..., X„, it follows that ~VxP(x) is the same 
as =(P(x,) A P(xy) A +++ A P(x,)), which is equivalent to =P(x,) V aP(x2) V + V aP(x,,) by De 
Morgan’s laws, and this is the same as 4x7P(x). Similarly, ~3xP(x) is the same as —(P(x,) V 
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TABLE 2 De Morgan’s Laws for Quantifiers. 


Equivalent Statement When Is Negation True? When False? 


VxaP(x) For every x, P(x) is false. There is an x for which 
P(x) is true. 
Ax7P(x) There is an x for which P(x) is true for every x. 
P(x) is false. 


P(x) V ++ V P(x,,)), which by De Morgan’s laws is equivalent to =P(x,) A aP) A +++ A aP(X,,), 
and this is the same as Vx7P(x). 


We illustrate the negation of quantified statements in Examples 20 and 21. 


What are the negations of the statements “There is an honest politician” and “All Americans eat 
cheeseburgers”? 


Solution: Let H(x) denote “x is honest.” Then the statement “There is an honest politician” is rep- 
resented by 3xH (x), where the domain consists of all politicians. The negation of this statement 
is 7AxH(x), which is equivalent to Vx~H (x). This negation can be expressed as “Every politician 
is dishonest.” (Note: In English, the statement “All politicians are not honest” is ambiguous. In 
common usage, this statement often means “Not all politicians are honest.” Consequently, we 
do not use this statement to express this negation.) 

Let C(x) denote “x eats cheeseburgers.” Then the statement “All Americans eat cheeseburg- 
ers” is represented by VxC(x), where the domain consists of all Americans. The negation of this 
statement is ~VxC(x), which is equivalent to dx-C(x). This negation can be expressed in sev- 
eral different ways, including “Some American does not eat cheeseburgers” and “There is an 
American who does not eat cheeseburgers.” < 


What are the negations of the statements Vx(x? > x) and Ax(x* = 2)? 


Solution: The negation of Yx(x? > x) is the statement 7Vx(x? > x), which is equivalent to 
3x(x? > x). This can be rewritten as 4x(x* < x). The negation of 4x(x* = 2) is the statement 
74x(x* = 2), which is equivalent to Vx>(x? = 2). This can be rewritten as Vx(x? # 2). The truth 
values of these statements depend on the domain. < 


We use De Morgan’s laws for quantifiers in Example 22. 
Show that =Vx(P(x) > Q(x)) and Ax(P(x) A 7Q(x)) are logically equivalent. 


Solution: By De Morgan’s law for universal quantifiers, we know that =Vx(P(x) > Q(x)) and 
Ax(-=(P(x) > Q(x))) are logically equivalent. By the fifth logical equivalence in Table 7 in Sec- 
tion 1.3, we know that =(P(x) > Q(x)) and P(x) A 7=Q(x) are logically equivalent for every x. 
Because we can substitute one logically equivalent expression for another in a logical equiva- 
lence, it follows that ~Vx(P(x) > Q(x)) and 3x(P(x) A =Q(x)) are logically equivalent. < 


1.4.10 Translating from English into Logical Expressions 


Translating sentences in English (or other natural languages) into logical expressions is a cru- 
cial task in mathematics, logic programming, artificial intelligence, software engineering, and 
many other disciplines. We began studying this topic in Section 1.1, where we used propositions 
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EXAMPLE 23 
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EXAMPLE 24 


to express sentences in logical expressions. In that discussion, we purposely avoided sentences 
whose translations required predicates and quantifiers. Translating from English to logical ex- 
pressions becomes even more complex when quantifiers are needed. Furthermore, there can 
be many ways to translate a particular sentence. (As a consequence, there is no “cookbook” 
approach that can be followed step by step.) We will use some examples to illustrate how to 
translate sentences from English into logical expressions. The goal in this translation is to pro- 
duce simple and useful logical expressions. In this section, we restrict ourselves to sentences 
that can be translated into logical expressions using a single quantifier; in the next section, we 
will look at more complicated sentences that require multiple quantifiers. 


Express the statement “Every student in this class has studied calculus” using predicates and 
quantifiers. 


Solution: First, we rewrite the statement so that we can clearly identify the appropriate quanti- 
fiers to use. Doing so, we obtain: 


“For every student in this class, that student has studied calculus.” 
Next, we introduce a variable x so that our statement becomes 
“For every student x in this class, x has studied calculus.” 


Continuing, we introduce C(x), which is the statement “x has studied calculus.” Consequently, 
if the domain for x consists of the students in the class, we can translate our statement as 
VxC(x). 

However, there are other correct approaches; different domains of discourse and other predi- 
cates can be used. The approach we select depends on the subsequent reasoning we want to carry 
out. For example, we may be interested in a wider group of people than only those in this class. 
If we change the domain to consist of all people, we will need to express our statement as 


“For every person x, if person x is a student in this class, then x has studied calculus.” 


If S(x) represents the statement that person x is in this class, we see that our statement can 
be expressed as Vx(S(x) > C(x)). [Caution! Our statement cannot be expressed as Vx(S(x) A 
C(x)) because this statement says that all people are students in this class and have studied 
calculus! ] 

Finally, when we are interested in the background of people in subjects besides calculus, 
we may prefer to use the two-variable quantifier Q(x, y) for the statement “Student x has stud- 
ied subject y.” Then we would replace C(x) by Q(x, calculus) in both approaches to obtain 
VxQ(x, calculus) or Vx($(x) > Q(x, calculus)). < 


In Example 23 we displayed different approaches for expressing the same statement us- 
ing predicates and quantifiers. However, we should always adopt the simplest approach that is 
adequate for use in subsequent reasoning. 


Express the statements “Some student in this class has visited Mexico” and “Every student in 
this class has visited either Canada or Mexico” using predicates and quantifiers. 


Solution: The statement “Some student in this class has visited Mexico” means that 
“There is a student in this class with the property that the student has visited Mexico.” 
We can introduce a variable x, so that our statement becomes 


“There is a student x in this class having the property that x has visited Mexico.” 
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We introduce M(x), which is the statement “x has visited Mexico.” If the domain for x consists 
of the students in this class, we can translate this first statement as IxM (x). 

However, if we are interested in people other than those in this class, we look at the statement 
a little differently. Our statement can be expressed as 


“There is a person x having the properties that x is a student in this class and x has visited 
Mexico.” 


In this case, the domain for the variable x consists of all people. We introduce S(x) to represent 
“x is a student in this class.” Our solution becomes 4x(S(x) A M(x)) because the statement is 
that there is a person x who is a student in this class and who has visited Mexico. [Caution! Our 
statement cannot be expressed as 4x(S(x) > M(x)), which is true when there is someone not in 
the class because, in that case, for such a person x, S(x) > M(x) becomes either F — T or F > 
F, both of which are true. ] 

Similarly, the second statement can be expressed as 


“For every x in this class, x has the property that x has visited Mexico or x has visited 
Canada.” 


(Note that we are assuming the inclusive, rather than the exclusive, or here.) We let C(x) be “x 
has visited Canada.” Following our earlier reasoning, we see that if the domain for x consists of 
the students in this class, this second statement can be expressed as Vx(C(x) V M(x)). However, 
if the domain for x consists of all people, our statement can be expressed as 


“For every person x, if x is a student in this class, then x has visited Mexico or x has visited 
Canada.” 


In this case, the statement can be expressed as Vx(S(x) > (C(x) V M(x))). 

Instead of using M(x) and C(x) to represent that x has visited Mexico and x has visited 
Canada, respectively, we could use a two-place predicate V(x, y) to represent “x has visited 
country y.” In this case, V(x, Mexico) and V(x, Canada) would have the same meaning as 
M(x) and C(x) and could replace them in our answers. If we are working with many state- 
ments that involve people visiting different countries, we might prefer to use this two-variable 
approach. Otherwise, for simplicity, we would stick with the one-variable predicates M(x) 
and C(x). < 


1.4.11 Using Quantifiers in System Specifications 


In Section 1.2 we used propositions to represent system specifications. However, many system 
specifications involve predicates and quantifications. This is illustrated in Example 25. 


Use predicates and quantifiers to express the system specifications “Every mail message larger 
than one megabyte will be compressed” and “Tf a user is active, at least one network link will 
be available.” 


Solution: Let S(m, y) be “Mail message m is larger than y megabytes,” where the variable x has 
the domain of all mail messages and the variable y is a positive real number, and let C(m) denote 
“Mail message m will be compressed.” Then the specification “Every mail message larger than 
one megabyte will be compressed” can be represented as Vm(S(m, 1) > C(m)). 

Let A(u) represent “User u is active,’ where the variable u has the domain of all users, 
let S(n, x) denote “Network link n is in state x,’ where n has the domain of all network 
links and x has the domain of all possible states for a network link. Then the specifica- 
tion “If a user is active, at least one network link will be available” can be represented by 
JuA(u) > InS(n, available). < 
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1.4.12 Examples from Lewis Carroll 


Lewis Carroll (really C. L. Dodgson writing under a pseudonym), the author of Alice in Wonder- 
land, is also the author of several works on symbolic logic. His books contain many examples 
of reasoning using quantifiers. Examples 26 and 27 come from his book Symbolic Logic; other 
examples from that book are given in the exercises at the end of this section. These examples 
illustrate how quantifiers are used to express various types of statements. 


Consider these statements. The first two are called premises and the third is called the conclu- 
sion. The entire set is called an argument. 


“All lions are fierce.” 
“Some lions do not drink coffee.” 
“Some fierce creatures do not drink coffee.” 


(In Section 1.6 we will discuss the issue of determining whether the conclusion is a valid con- 
sequence of the premises. In this example, it is.) Let P(x), Q(x), and R(x) be the statements “x is 
a lion,” “x is fierce,” and “x drinks coffee,” respectively. Assuming that the domain consists of 


all creatures, express the statements in the argument using quantifiers and P(x), Q(x), and R(x). 


Solution: We can express these statements as 


Vx(P(x) > Q(x)). 
Ax(P(x) A aR). 
3x(O(x) A 7R(x)). 


Notice that the second statement cannot be written as 4x(P(x) > —R(x)). The reason is that 
P(x) > 7R(x) is true whenever x is not a lion, so that 3x(P(x) > —=R(x)) is true as long as there 
is at least one creature that is not a lion, even if every lion drinks coffee. Similarly, the third 
statement cannot be written as 


KOE) > R). 4 


Consider these statements, of which the first three are premises and the fourth is a valid con- 
clusion. 


“All hummingbirds are richly colored.” 

“No large birds live on honey.” 

“Birds that do not live on honey are dull in color.” 
“Hummingbirds are small.” 


CHARLES LUTWIDGE DODGSON (1832-1898) We know Charles Dodgson as Lewis Carroll—the 
pseudonym he used in his literary works. Dodgson, the son of a clergyman, was the third of 11 children, all of 
whom stuttered. He was uncomfortable in the company of adults and is said to have spoken without stuttering 
only to young girls, many of whom he entertained, corresponded with, and photographed (sometimes in poses 
that today would be considered inappropriate). Although attracted to young girls, he was extremely puritan- 
ical and religious. His friendship with the three young daughters of Dean Liddell led to his writing Alice in 
Wonderland, which brought him money and fame. 

Dodgson graduated from Oxford in 1854 and obtained his master of arts degree in 1857. He was appointed 
lecturer in mathematics at Christ Church College, Oxford, in 1855. He was ordained in the Church of England 
in 1861 but never practiced his ministry. His writings published under this real name include articles and books 
on geometry, determinants, and the mathematics of tournaments and elections. (He also used the pseudonym 
Lewis Carroll for his many works on recreational logic.) 
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Let P(x), Q(x), R(x), and S(x) be the statements “x is a hummingbird,” “x is large,” “x lives on 
honey,” and “x is richly colored,” respectively. Assuming that the domain consists of all birds, 
express the statements in the argument using quantifiers and P(x), Q(x), R(x), and S(x). 


Solution: We can express the statements in the argument as 


Vx(P(x) > S(x)). 
74x(O(x) A R(x)). 
Vx(7R(x) > aS(x)). 
Vx(P(x) > 7Q(x)). 


(Note we have assumed that “small” is the same as “not large” and that “dull in color” is the 
same as “not richly colored.” To show that the fourth statement is a valid conclusion of the first 
three, we need to use rules of inference that will be discussed in Section 1.6.) < 


1.4.13 Logic Programming 


An important type of programming language is designed to reason using the rules of predicate 
logic. Prolog (from Programming in Logic), developed in the 1970s by computer scientists 
working in the area of artificial intelligence, is an example of such a language. Prolog programs 
include a set of declarations consisting of two types of statements, Prolog facts and Prolog 
rules. Prolog facts define predicates by specifying the elements that satisfy these predicates. 
Prolog rules are used to define new predicates using those already defined by Prolog facts. 
Example 28 illustrates these notions. 


Consider a Prolog program given facts telling it the instructor of each class and in which 
classes students are enrolled. The program uses these facts to answer queries concerning the 
professors who teach particular students. Such a program could use the predicates instruc- 
tor(p, c) and enrolled(s, c) to represent that professor p is the instructor of course c and that 
student s is enrolled in course c, respectively. For example, the Prolog facts in such a program 
might include: 


instructor(chan,math273) 
instructor(patel ,ee222) 
instructor(grossman,cs301) 
enrolled(kevin,math273) 
enrolled(juana,ee222) 
enrolled(juana,cs301) 
enrolled(kiko,math273) 
enrolled(kiko,cs301) 


(Lowercase letters have been used for entries because Prolog considers names beginning with 
an uppercase letter to be variables.) 

A new predicate teaches(p, s), representing that professor p teaches student s, can be defined 
using the Prolog rule 


teaches(P,S) :- instructor(P,C), enrolled(S,C) 


which means that teaches(p, s) is true if there exists a class c such that professor p is the in- 
structor of class c and student s is enrolled in class c. (Note that a comma is used to represent a 
conjunction of predicates in Prolog. Similarly, a semicolon is used to represent a disjunction of 
predicates.) 
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Prolog answers queries using the facts and rules it is given. For example, using the facts 


and rules listed, the query 


?enrolled(kevin,math273) 


produces the response 


yes 


because the fact enrolled(kevin, math273) was provided as input. The query 


?enrolled(X,math273) 


produces the response 


kevin 
kiko 


To produce this response, Prolog determines all possible values of X for which 
enrolled(X, math273) has been included as a Prolog fact. Similarly, to find all the professors 
who are instructors in classes being taken by Juana, we use the query 


?teaches(X, juana) 


This query returns 


patel 
grossman 
Exercises 

1. Let P(x) denote the statement “x < 4.” What are these 
truth values? 

a) P(O) b) P(4) c) P(6) 

2. Let P(x) be the statement “The word x contains the 
letter a.” What are these truth values? 
a) P(orange) b) Pdemon) 

c) P(true) d) P(false) 

3. Let Q(x, y) denote the statement “x is the capital of y.” 
What are these truth values? 

a) Q(Denver, Colorado) 
b) Q(Detroit, Michigan) 
c) Q(Massachusetts, Boston) 
d) Q(New York, New York) 

4. State the value of x after the statement if P(x) then x := 1 
is executed, where P(x) is the statement “x > 1,” if the 
value of x when this statement is reached is 
a) x=0. b) x=1. 

c) x=2. 
5. Let P(x) be the statement “x spends more than five hours 


every weekday in class,” where the domain for x consists 
of all students. Express each of these quantifications in 
English. 


a) AxP(x) 
co) Ax7P(x) 


b) VxP(x) 
d) Vx7>P(x) 


. Let N(x) be the statement “x has visited North Dakota,” 


where the domain consists of the students in your school. 
Express each of these quantifications in English. 

a) AxN(x) b) VxN(x) c) 7AxN(x) 

d) Ax7N(x) e) 7AVxN(x) f) VxN(x) 


. Translate these statements into English, where C(x) is “x 


is a comedian” and F(x) is “x is funny” and the domain 
consists of all people. 
a) Vx(C(x) > F(x) 
ce) Ax(C(x) > F(x)) 


b) Vx(C(x) A F(x)) 
d) Ax(C(x) A F(x) 


. Translate these statements into English, where R(x) is “x 


is a rabbit” and H(x) is “x hops” and the domain consists 
of all animals. 

a) Vx(R(x) > H(x)) 
c) Ax(R@) > A(x) 


b) Vx(R(x) A H(x)) 
d) 4x(R(x) A H(x)) 


. Let P(x) be the statement “x can speak Russian” and let 


Q(x) be the statement “x knows the computer language 
C++.” Express each of these sentences in terms of P(x), 
Q(x), quantifiers, and logical connectives. The domain 
for quantifiers consists of all students at your school. 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


a) There is a student at your school who can speak Rus- 
sian and who knows C++. 

b) There is a student at your school who can speak Rus- 
sian but who doesn’t know C++. 

c) Every student at your school either can speak Russian 
or knows C++. 

d) No student at your school can speak Russian or knows 
C++. 

Let C(x) be the statement “x has a cat,” let D(x) be the 

statement “x has a dog,” and let F(x) be the statement “x 

has a ferret.” Express each of these statements in terms 

of C(x), D(x), F(x), quantifiers, and logical connectives. 

Let the domain consist of all students in your class. 

a) A student in your class has a cat, a dog, and a ferret. 

b) All students in your class have a cat, a dog, or a ferret. 

c) Some student in your class has a cat and a ferret, but 
not a dog. 

d) No student in your class has a cat, a dog, and a ferret. 

e) For each of the three animals, cats, dogs, and ferrets, 
there is a student in your class who has this animal as 
a pet. 

Let P(x) be the statement “x = x?.” If the domain consists 

of the integers, what are these truth values? 

a) P(O) b) Pd) c) P(2) 

d) P(-1) e) AxP(x) f) VxP(x) 

Let Q(x) be the statement “x + 1 > 2x.” If the domain 

consists of all integers, what are these truth values? 


a) Q(0) b) Q-)) ce) Q0) 
d) AxQ(x) e) VxQ(x) £) Doga) 
g) Vx7Q(x) 


Determine the truth value of each of these statements if 
the domain consists of all integers. 

a) Yn(n+ 1 >n) b) an(2n = 3n) 

c) an(n = —n) d) Yn(3n < 4n) 

Determine the truth value of each of these statements if 
the domain consists of all real numbers. 

a) dx? = —1) b) dx(x* < x?) 

eo) Vx((—x)? = x?) d) Vx(2x > x) 

Determine the truth value of each of these statements if 
the domain for all variables consists of all integers. 

a) Yn(n? > 0) b) dn(n? = 2) 

ce) Va(n? > n) d) An(n? < 0) 

Determine the truth value of each of these statements 
if the domain of each variable consists of all real num- 
bers. 

a) Ix = 2) b) dx? = —1) 

co) Vx0? +2> 1) d) Vx? + x) 

Suppose that the domain of the propositional function 
P(x) consists of the integers 0, 1, 2, 3, and 4. Write out 
each of these propositions using disjunctions, conjunc- 
tions, and negations. 


a) AxP(x) b) VxP(x) 
d) Vx7P(x) e) 7dxP(x) 


ce) Ax7AP(x) 
f) 7AVxP(x) 
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Suppose that the domain of the propositional function 
P(x) consists of the integers —2, —1, 0, 1, and 2. Write 
out each of these propositions using disjunctions, con- 
junctions, and negations. 

a) IxP(x) b) YxP(x) c) IaP(x) 

d) Vx-=P(x) e) =73xP(x) f) =VxP(x) 
Suppose that the domain of the propositional function 
P(x) consists of the integers 1, 2, 3, 4, and 5. Express 
these statements without using quantifiers, instead using 
only negations, disjunctions, and conjunctions. 

a) IxP(x) b) YxP(x) 

c) 74xP(@) d) =VxP(x) 

e) Vx((x # 3) > P(x)) v Ax7P(x) 

Suppose that the domain of the propositional function 
P(x) consists of —5, —3, —1, 1, 3, and 5. Express these 
statements without using quantifiers, instead using only 
negations, disjunctions, and conjunctions. 

a) AxP(x) b) VxP(@) 

c) Vx(Qx # 1) > PŒ) 

d) Ax((x = 0) A PQ) 

e) Ax(4P(x)) A Yx((x < 0) > P@)) 

For each of these statements find a domain for which the 
statement is true and a domain for which the statement is 
false. 

a) Everyone is studying discrete mathematics. 

b) Everyone is older than 21 years. 

c) Every two people have the same mother. 

d) No two different people have the same grandmother. 
For each of these statements find a domain for which the 
statement is true and a domain for which the statement is 
false. 

a) Everyone speaks Hindi. 

b) There is someone older than 21 years. 

c) Every two people have the same first name. 

d) Someone knows more than two other people. 
Translate in two ways each of these statements into logi- 
cal expressions using predicates, quantifiers, and logical 
connectives. First, let the domain consist of the students 
in your class and second, let it consist of all people. 

a) Someone in your class can speak Hindi. 


b) Everyone in your class is friendly. 

c) There is a person in your class who was not born in 
California. 

d) A student in your class has been in a movie. 

e) No student in your class has taken a course in logic 


programming. 
Translate in two ways each of these statements into logi- 
cal expressions using predicates, quantifiers, and logical 
connectives. First, let the domain consist of the students 
in your class and second, let it consist of all people. 
a) Everyone in your class has a cellular phone. 
b) Somebody in your class has seen a foreign movie. 
c) There is a person in your class who cannot swim. 
d) All students in your class can solve quadratic equa- 
tions. 
e) Some student in your class does not want to be rich. 
Translate each of these statements into logical expres- 
sions using predicates, quantifiers, and logical connec- 
tives. 
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a) No one is perfect. 

b) Not everyone is perfect. 

c) All your friends are perfect. 

d) At least one of your friends is perfect. 

e) Everyone is your friend and is perfect. 

f) Not everybody is your friend or someone is not per- 
fect. 


Translate each of these statements into logical expres- 
sions in three different ways by varying the domain and 
by using predicates with one and with two variables. 


a) Someone in your school has visited Uzbekistan. 
b) Everyone in your class has studied calculus and C++. 


c) No one in your school owns both a bicycle and a mo- 
torcycle. 


d) There is a person in your school who is not happy. 
e) Everyone in your school was born in the twentieth 
century. 


Translate each of these statements into logical expres- 
sions in three different ways by varying the domain and 
by using predicates with one and with two variables. 


a) A student in your school has lived in Vietnam. 

b) There is a student in your school who cannot speak 
Hindi. 

c) A student in your school knows Java, Prolog, and 
C++. 

d) Everyone in your class enjoys Thai food. 

e) Someone in your class does not play hockey. 


Translate each of these statements into logical expres- 
sions using predicates, quantifiers, and logical connec- 
tives. 


a) Something is not in the correct place. 


b) All tools are in the correct place and are in excellent 
condition. 


c) Everything is in the correct place and in excellent 
condition. 


d) Nothing is in the correct place and is in excellent con- 
dition. 

e) One of your tools is not in the correct place, but it is 
in excellent condition. 


Express each of these statements using logical operators, 
predicates, and quantifiers. 


a) Some propositions are tautologies. 

b) The negation of a contradiction is a tautology. 

c) The disjunction of two contingencies can be a tautol- 
ogy. 

d) The conjunction of two tautologies is a tautology. 


Suppose the domain of the propositional function P(x, y) 
consists of pairs x and y, where x is 1, 2, or 3 and yis 1, 2, 
or 3. Write out these propositions using disjunctions and 
conjunctions. 
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a) Ax P(x, 3) b) Vy Pd, y) 

c) dy7P(, y) d) Yx=P(x, 2) 

Suppose that the domain of Q(x, y, z) consists of triples 

xX, y, Zz, Where x = 0, 1, or 2, y= 0 or 1, and z=0 or 1. 

Write out these propositions using disjunctions and con- 

junctions. 

a) YyQ(0, y, 0) b) 4xQ(x 1, 1) 

c) 3z7Q(0, 0, z) d) 3x7Q(x, 0, 1) 

Express each of these statements using quantifiers. Then 

form the negation of the statement so that no negation is 

to the left of a quantifier. Next, express the negation in 

simple English. (Do not simply use the phrase “It is not 

the case that.”) 

a) All dogs have fleas. 

b) There is a horse that can add. 

c) Every koala can climb. 

d) No monkey can speak French. 

e) There exists a pig that can swim and catch fish. 

Express each of these statements using quantifiers. Then 

form the negation of the statement, so that no negation 

is to the left of a quantifier. Next, express the negation in 

simple English. (Do not simply use the phrase “It is not 

the case that.”) 

a) Some old dogs can learn new tricks. 

b) No rabbit knows calculus. 

c) Every bird can fly. 

d) There is no dog that can talk. 

e) There is no one in this class who knows French and 
Russian. 

Express the negation of these propositions using quanti- 

fiers, and then express the negation in English. 

a) Some drivers do not obey the speed limit. 

b) All Swedish movies are serious. 

c) No one can keep a secret. 

d) There is someone in this class who does not have a 
good attitude. 

Express the negation of each of these statements in terms 

of quantifiers without using the negation symbol. 

a) Vx(x > 1) 

b) Yx(x < 2) 

c) Ax(x > 4) 

d) Ax(x < 0) 

e) Vx(x < —1) V@ > 2)) 

f) ax(a<4)V@>7)) 

Express the negation of each of these statements in terms 

of quantifiers without using the negation symbol. 

a) Vx(—2 < x < 3) 

b) Vx(0 < x < 5) 

c) 3x(—4 <x <1) 

d) 3x(—5 < x < —1) 

Find a counterexample, if possible, to these universally 

quantified statements, where the domain for all variables 

consists of all integers. 

a) Yx > x) 

b) Yxx>0vx<0) 

c) Vx(x = 1) 


38. 


39. 


Find a counterexample, if possible, to these universally 
quantified statements, where the domain for all variables 
consists of all real numbers. 
a) Vx £ x) 

c) Vx(|x| > 0) 

Express each of these statements using predicates and 
quantifiers. 


b) Yx? # 2) 


a) A passenger on an airline qualifies as an elite flyer if 
the passenger flies more than 25,000 miles in a year 
or takes more than 25 flights during that year. 

b) A man qualifies for the marathon if his best previ- 
ous time is less than 3 hours and a woman qualifies 
for the marathon if her best previous time is less than 
3.5 hours. 

c) A student must take at least 60 course hours, or at least 
45 course hours and write a master’s thesis, and re- 
ceive a grade no lower than a B in all required courses, 
to receive a master’s degree. 

d) There is a student who has taken more than 21 credit 
hours in a semester and received all A’s. 


Exercises 40-44 deal with the translation between system 
specification and logical expressions involving quantifiers. 


40. 


41. 


42. 


43. 


Translate these system specifications into English, where 

the predicate S(x, y) is “x is in state y” and where the do- 

main for x and y consists of all systems and all possible 
states, respectively. 

a) AxS(x, open) 

b) Yx(S(x, malfunctioning) V S(x, diagnostic)) 

c) AxS(x, open) v AxS(x, diagnostic) 

d) Ax-S(x, available) 

e) Vx7S(x, working) 

Translate these specifications into English, where F(p) is 

“Printer p is out of service,” B(p) is “Printer p is busy,” 

LQG) is “Print job j is lost,’ and Q(j) is “Print job j is 

queued.” 

a) Ap(F(p) A Bip)) > ALY) 

b) VpB(p) > AQ) 

©) AQ) ALG) > Apr) 

d) (VpB) A VjQQ)) > ILG) 

Express each of these system specifications using predi- 

cates, quantifiers, and logical connectives. 

a) When there is less than 30 megabytes free on the hard 
disk, a warning message is sent to all users. 

b) No directories in the file system can be opened and 
no files can be closed when system errors have been 
detected. 

c) The file system cannot be backed up if there is a user 
currently logged on. 

d) Video on demand can be delivered when there are at 
least 8 megabytes of memory available and the con- 
nection speed is at least 56 kilobits per second. 

Express each of these system specifications using predi- 

cates, quantifiers, and logical connectives. 

a) At least one mail message, among the nonempty set 
of messages, can be saved if there is a disk with more 
than 10 kilobytes of free space. 


44. 


45. 


46. 


47. 
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b) Whenever there is an active alert, all queued messages 
are transmitted. 

c) The diagnostic monitor tracks the status of all systems 
except the main console. 

d) Each participant on the conference call whom the host 
of the call did not put on a special list was billed. 
Express each of these system specifications using predi- 

cates, quantifiers, and logical connectives. 

a) Every user has access to an electronic mailbox. 

b) The system mailbox can be accessed by everyone in 
the group if the file system is locked. 

c) The firewall is in a diagnostic state only if the proxy 
server is in a diagnostic state. 

d) At least one router is functioning normally if the 
throughput is between 100 kbps and 500 kbps and the 
proxy server is not in diagnostic mode. 

Determine whether Vx(P(x) > Q(x)) and VxP(x) > 

VxQ(x) are logically equivalent. Justify your answer. 

Determine whether Vx(P(x) @ Q(x)) and Vx P(x) o 

VxQ(x) are logically equivalent. Justify your answer. 

Show that 4x(P(x) V Q(x)) and 3xP(x) V IxQ(x) are log- 

ically equivalent. 


Exercises 48-51 establish rules for null quantification that 
we can use when a quantified variable does not appear in part 
of a statement. 


48. 


49. 


50. 


51. 


52. 


53. 


54. 


Establish these logical equivalences, where x does not oc- 
cur as a free variable in A. Assume that the domain is 
nonempty. 

a) (VxP(x)) VA = Yx(P@Œ) V A) 

b) (AxPQ@)) VA = Ax(P(®) V A) 

Establish these logical equivalences, where x does not oc- 
cur as a free variable in A. Assume that the domain is 
nonempty. 

a) (VxP(x)) AA = Vx(P(x) AA) 

b) (AxPQ@)) AA = Ax(P(X) AA) 

Establish these logical equivalences, where x does not oc- 
cur as a free variable in A. Assume that the domain is 
nonempty. 

a) Vx(A > P(x) =A > VxP(x) 

b) 3x(A > PQ@)) =A > AxP(x) 

Establish these logical equivalences, where x does not oc- 
cur as a free variable in A. Assume that the domain is 
nonempty. 

a) Vx(P(x) > A) = 3xP(x) > A 

b) 3x(P(x) > A) = VxP(x) > A 

Show that VxP(x) V YxQ(x) and Vx(P(x) v Q(x)) are not 
logically equivalent. 

Show that 3xP(x) A IxQ(x) and Ix(P(x) A Q(x)) are not 
logically equivalent. 


As mentioned in the text, the notation 3!xP(x) denotes 
“There exists a unique x such that P(x) is true.” 

If the domain consists of all integers, what are the truth 

values of these statements? 

a) 3Jlx(x > 1) 

c) Alx@ +3 = 2x) 


b) Ix = 1) 
d) Alex =x+41) 
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What are the truth values of these statements? 

a) A!lxP(x) > AxP(x) 

b) VxP(x) > A!xP(x) 

ce) AlxP(x) > AVxP(x) 

Write out 4!xP(x), where the domain consists of the inte- 
gers 1, 2, and 3, in terms of negations, conjunctions, and 
disjunctions. 

Given the Prolog facts in Example 28, what would Prolog 
return given these queries? 

a) ?instructor(chan,math273) 

b) ?instructor(patel,cs301) 

c) ?enrolled(X,cs301) 

d) ?enrolled(kiko,Y) 

e) ?teaches(grossman,yY) 

Given the Prolog facts in Example 28, what would Prolog 
return when given these queries? 

a) ?enrolled(kevin,ee222) 

b) ?enrolled(kiko,math273) 

c) ?instructor(grossman, xX) 

d) ?instructor(X,cs301) 

e) ?teaches(X, kevin) 

Suppose that Prolog facts are used to define the predicates 
mother(M, Y) and father(F, X), which represent that M is 
the mother of Y and F is the father of X, respectively. 
Give a Prolog rule to define the predicate sibling(X, Y), 
which represents that X and Y are siblings (that is, have 
the same mother and the same father). 

Suppose that Prolog facts are used to define the pred- 
icates mother(M, Y) and father(F, X), which represent 
that M is the mother of Y and F is the father of X, re- 
spectively. Give a Prolog rule to define the predicate 
grandfather(X, Y), which represents that X is the grand- 
father of Y. [Hint: You can write a disjunction in Prolog 
either by using a semicolon to separate predicates or by 
putting these predicates on separate lines. ] 


Exercises 61-64 are based on questions found in the book 
Symbolic Logic by Lewis Carroll. 


61. 


Let P(x), Q(x), and R(x) be the statements “x is a profes- 
sor,” “x is ignorant,” and “x is vain,” respectively. Express 
each of these statements using quantifiers; logical con- 
nectives; and P(x), Q(x), and R(x), where the domain 


consists of all people. 


Nested Quantifiers 


62. 


63 


64 


a) No professors are ignorant. 

b) All ignorant people are vain. 

c) No professors are vain. 

d) Does (c) follow from (a) and (b)? 

Let P(x), Q(x), and R(x) be the statements “x is a clear 

explanation,” “x is satisfactory,’ and “x is an excuse,” 

respectively. Suppose that the domain for x consists of 
all English text. Express each of these statements us- 
ing quantifiers, logical connectives, and P(x), Q(x), and 

R(x). 

a) All clear explanations are satisfactory. 

b) Some excuses are unsatisfactory. 

c) Some excuses are not clear explanations. 

*d) Does (c) follow from (a) and (b)? 

. Let P(x), Q(x), R(x), and S(x) be the statements “x is a 
baby,” “x is logical,” “x is able to manage a crocodile,” 
and “x is despised,” respectively. Suppose that the domain 
consists of all people. Express each of these statements 
using quantifiers; logical connectives; and P(x), Q(x), 
R(x), and S(x). 

a) Babies are illogical. 
b) Nobody is despised who can manage a crocodile. 
c) Illogical persons are despised. 
d) Babies cannot manage crocodiles. 
*e) Does (d) follow from (a), (b), and (c)? If not, is there 
a correct conclusion? 

. Let P(x), Q(x), R(x), and S(x) be the statements “x is a 
duck,” “x is one of my poultry,” “x is an officer,” and “x 
is willing to waltz,’ respectively. Express each of these 
statements using quantifiers; logical connectives; and 
P(x), Q(x), R(x), and S(x). 

a) No ducks are willing to waltz. 
b) No officers ever decline to waltz. 
c) All my poultry are ducks. 

d) My poultry are not officers. 


*e) Does (d) follow from (a), (b), and (c)? If not, is there 
a correct conclusion? 


1.5.1 Introduction 


In Section 1.4 we defined the existential and universal quantifiers and showed how they can be 
used to represent mathematical statements. We also explained how they can be used to translate 
English sentences into logical expressions. However, in Section 1.4 we avoided nested quanti- 
fiers, where one quantifier is within the scope of another, such as 


Vxdy(x + y = 0). 


EXAMPLE 1 
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EXAMPLE 2 
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Note that everything within the scope of a quantifier can be thought of as a propositional func- 
tion. For example, 


Vxdy(x + y = 0) 


is the same thing as VxQ(x), where Q(x) is 3yP(x, y), where P(x, y) is x + y = 0. 

Nested quantifiers commonly occur in mathematics and computer science. Although nested 
quantifiers can sometimes be difficult to understand, the rules we have already studied in 
Section 1.4 can help us use them. In this section we will gain experience working with nested 
quantifiers. We will see how to use nested quantifiers to express mathematical statements such 
as “The sum of two positive integers is always positive.” We will show how nested quantifiers 
can be used to translate English sentences such as “Everyone has exactly one best friend” into 
logical statements. Moreover, we will gain experience working with the negations of statements 
involving nested quantifiers. 


1.5.2 Understanding Statements Involving Nested Quantifiers 


To understand statements involving nested quantifiers, we need to unravel what the quantifiers 
and predicates that appear mean. This is illustrated in Examples 1 and 2. 


Assume that the domain for the variables x and y consists of all real numbers. The statement 
VxVya +y =y+x) 


says that x + y = y + x for all real numbers x and y. This is the commutative law for addition of 
real numbers. Likewise, the statement 


Vxdy@ +y = 0) 


says that for every real number x there is a real number y such that x + y = 0. This states that 
every real number has an additive inverse. Similarly, the statement 


VaVy2(x + (y + z) = (x+y) +2) 


is the associative law for addition of real numbers. < 


Translate into English the statement 
VaVy((x > 0) A (y < 0) > (xy < 0)), 
where the domain for both variables consists of all real numbers. 


Solution: This statement says that for every real number x and for every real number y, if x > 0 
and y < 0, then xy < 0. That is, this statement says that for real numbers x and y, if x is positive 
and y is negative, then xy is negative. This can be stated more succinctly as “The product of a 
positive real number and a negative real number is always a negative real number.” < 


THINKING OF QUANTIFICATION AS LOOPS In working with quantifications of more 
than one variable, it is sometimes helpful to think in terms of nested loops. (If there are infinitely 
many elements in the domain of some variable, we cannot actually loop through all values. 
Nevertheless, this way of thinking is helpful in understanding nested quantifiers.) For example, 
to see whether VxVyP(x, y) is true, we loop through the values for x, and for each x we loop 
through the values for y. If we find that for all values of x that P(x, y) is true for all values of y, 
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EXAMPLE 3 
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we have determined that VxVyP(x, y) is true. If we ever hit a value x for which we hit a value y 
for which P(x, y) is false, we have shown that VxVyP(, y) is false. 

Similarly, to determine whether Yx3yP(x, y) is true, we loop through the values for x. For 
each x we loop through the values for y until we find a y for which P(x, y) is true. If for every x we 
hit such a y, then Yx3yP(x, y) is true; if for some x we never hit such a y, then VxdyP(, y) is false. 

To see whether 4xVyP(x, y) is true, we loop through the values for x until we find an x for 
which P(x, y) is always true when we loop through all values for y. Once we find such an x, we 
know that 4xVyP(a, y) is true. If we never hit such an x, then we know that 4xVyP(, y) is false. 

Finally, to see whether 3x3yP(x, y) is true, we loop through the values for x, where for each 
x we loop through the values for y until we hit an x for which we hit a y for which P(x, y) is true. 
The statement 3x3yP(x, y) is false only if we never hit an x for which we hit a y such that P(x, y) 
is true. 


1.5.3 The Order of Quantifiers 


Many mathematical statements involve multiple quantifications of propositional functions in- 
volving more than one variable. It is important to note that the order of the quantifiers is impor- 
tant, unless all the quantifiers are universal quantifiers or all are existential quantifiers. 

These remarks are illustrated by Examples 3-5. 


Let P(x, y) be the statement “x + y = y + x.” What are the truth values of the quantifications 
VxVyP(x, y) and VyVxP(x, y), where the domain for all variables consists of all real numbers? 


Solution: The quantification 
VxVyP(x, y) 
denotes the proposition 
“For all real numbers x, for all real numbers y, x + y = y + x.” 


Because P(x, y) is true for all real numbers x and y (it is the commutative law for addition, which 
is an axiom for the real numbers—see Appendix 1), the proposition VxVyP(x, y) is true. Note that 
the statement VyVxP(x, y) says “For all real numbers y, for all real numbers x, x + y = y+ x.” 
This has the same meaning as the statement “For all real numbers x, for all real numbers y, 
x+y = y +x.” That is, VxVyP(, y) and VyVxP(x, y) have the same meaning, and both are true. 
This illustrates the principle that the order of nested universal quantifiers in a statement without 
other quantifiers can be changed without changing the meaning of the quantified statement. <4 


Let Q(x, y) denote “x + y = 0.” What are the truth values of the quantifications SyVxQ(a, y) and 
VxdyQ(%, y), where the domain for all variables consists of all real numbers? 


Solution: The quantification 
AyVxQ(, y) 
denotes the proposition 
“There is a real number y such that for every real number x, Q(x, y).” 
No matter what value of y is chosen, there is only one value of x for which x + y = 0. Because 


there is no real number y such that x + y = 0 for all real numbers x, the statement SyVxQ(x, y) 
is false. 


Be careful with the 
order of existential and 
universal quantifiers! 


EXAMPLE 5 
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The quantification 


VxAyQ(x, y) 


denotes the proposition 
“For every real number x there is a real number y such that Q(x, y).” 


Given a real number x, there is a real number y such that x + y = 0; namely, y = —x. Hence, the 
statement VxAyQ(x, y) is true. < 


Example 4 illustrates that the order in which quantifiers appear makes a difference. The 
statements 4yVxP(x, y) and VxdyP(, y) are not logically equivalent. The statement SyVxP(x, y) 
is true if and only if there is a y that makes P(x, y) true for every x. So, for this statement to 
be true, there must be a particular value of y for which P(x, y) is true regardless of the choice 
of x. On the other hand, Yx3yP(x, y) is true if and only if for every value of x there is a value 
of y for which P(x, y) is true. So, for this statement to be true, no matter which x you choose, 
there must be a value of y (possibly depending on the x you choose) for which P(x, y) is true. 
In other words, in the second case, y can depend on x, whereas in the first case, y is a constant 
independent of x. 

From these observations, it follows that if SyVxP(x, y) is true, then VxdyP(x, y) must also 
be true. However, if VxdyP(x, y) is true, it is not necessary for JyVxP(, y) to be true. (See 
Supplementary Exercises 30 and 31.) 

Table 1 summarizes the meanings of the different possible quantifications involving two 
variables. 

Quantifications of more than two variables are also common, as Example 5 illustrates. 


” 


Let Q(x, y,z) be the statement “x+y =z.’ What are the truth values of the statements 
VxVydzO(, y, z) and AzVxVyQQ, y, z), where the domain of all variables consists of all real 
numbers? 


Solution: Suppose that x and y are assigned values. Then, there exists a real number z such that 
x+y = z. Consequently, the quantification 


VxVylzO(x, y, z), 
which is the statement 


“For all real numbers x and for all real numbers y there is a real number z such that 
x+y=z” 


TABLE 1 Quantifications of Two Variables. 


YxYyP(x, y) P(x, y) is true for every pair x, y. There is a pair x, y for 
VyVxP(x, y) which P(x, y) is false. 


VxdyP(, y) For every x there is a y for There is an x such that 
which P(x, y) is true. P(x, y) is false for every y 


AxVyP(x, y) There is an x for which P(x, y) For every x there is a y for 

is true for every y. which P(x, y) is false 
AxdyP(x, y) There is a pair x, y for which P(x, y) is false for every 
J3yaxP(x, y) P(x, y) is true. pair x, y. 
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is true. The order of the quantification here is important, because the quantification 
AzVxVyQ(x, y, z), 


which is the statement 


“There is a real number z such that for all real numbers x and for all real numbers y it is true 
that x +y = z,” 


is false, because there is no value of z that satisfies the equation x + y = z for all values of x 
and y. < 


1.5.4 Translating Mathematical Statements into Statements 
Involving Nested Quantifiers 


Mathematical statements expressed in English can be translated into logical expressions, as 
Examples 6-8 show. 


Translate the statement “The sum of two positive integers is always positive” into a logical 
expression. 


Solution: To translate this statement into a logical expression, we first rewrite it so that the 
implied quantifiers and a domain are shown: “For every two integers, if these integers are both 
positive, then the sum of these integers is positive.” Next, we introduce the variables x and y to 
obtain “For all positive integers x and y, x + y is positive.” Consequently, we can express this 
statement as 


VxVy((x > 0) A O > 0) = &+y > 0), 
where the domain for both variables consists of all integers. Note that we could also translate 
this using the positive integers as the domain. Then the statement “The sum of two positive 


integers is always positive” becomes “For every two positive integers, the sum of these integers 
is positive.” We can express this as 


VxVy(x + y > 0), 


where the domain for both variables consists of all positive integers. < 


Translate the statement “Every real number except zero has a multiplicative inverse.” (A mul- 
tiplicative inverse of a real number x is a real number y such that xy = 1.) 


Solution: We first rewrite this as “For every real number x except zero, x has a multiplicative 


inverse.” We can rewrite this as “For every real number x, if x Æ 0, then there exists a real 
number y such that xy = 1.” This can be rewritten as 


Va((x # 0) > Iw = 1)). < 


One example that you may be familiar with is the concept of limit, which is important in 
calculus. 


EXAMPLE 8 


EXAMPLE 9 


EXAMPLE 10 
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(Requires calculus) Use quantifiers to express the definition of the limit of a real-valued func- 
tion f(x) of a real variable x at a point a in its domain. 


Solution: Recall that the definition of the statement 


lin f(x) = L 
is: For every real number e > 0 there exists a real number ô > 0 such that |[f(x) —L| < e 
whenever 0 < |x — a| < ô. This definition of a limit can be phrased in terms of quantifiers by 


Vedd6Vx(0 < |x-—al < ô > [fŒ -L| < e€), 


where the domain for the variables ô and e consists of all positive real numbers and for x consists 
of all real numbers. 
This definition can also be expressed as 


Ve>046>0 Vx(0 < |x-aļ| <6 > [f@) -L| <e) 


when the domain for the variables e and 6 consists of all real numbers, rather than just the 
positive real numbers. [Here, restricted quantifiers have been used. Recall that Vx > 0 P(x) means 
that for all x with x >0, P(x) is true.] < 


1.5.5 Translating from Nested Quantifiers into English 


Expressions with nested quantifiers expressing statements in English can be quite complicated. 
The first step in translating such an expression is to write out what the quantifiers and predicates 
in the expression mean. The next step is to express this meaning in a simpler sentence. This 
process is illustrated in Examples 9 and 10. 


Translate the statement 
Vx(C(x) V Ay(C(y) A Fæ, y))) 


into English, where C(x) is “x has a computer,” F(x, y) is “x and y are friends,” and the domain 
for both x and y consists of all students in your school. 


Solution: The statement says that for every student x in your school, x has a computer or there 
is a student y such that y has a computer and x and y are friends. In other words, every student 
in your school has a computer or has a friend who has a computer. < 


Translate the statement 
AxVyV2((FO, y) A FO, z) A O #2) > “FQ, z)) 


into English, where F(a, b) means a and b are friends and the domain for x, y, and z consists of 
all students in your school. 


Solution: We first examine the expression (F(x, y) A F(x, z) A O # Z)) > aF (y, z). This expres- 
sion says that if students x and y are friends, and students x and z are friends, and furthermore, 
if y and z are not the same student, then y and z are not friends. It follows that the original state- 
ment, which is triply quantified, says that there is a student x such that for all students y and all 
students z other than y, if x and y are friends and x and z are friends, then y and z are not friends. 
In other words, there is a student none of whose friends are also friends with each other. < 
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1.5.6 Translating English Sentences into Logical Expressions 


In Section 1.4 we showed how quantifiers can be used to translate sentences into logical expres- 
sions. However, we avoided sentences whose translation into logical expressions required the 
use of nested quantifiers. We now address the translation of such sentences. 


Express the statement “If a person is female and is a parent, then this person is someone’s 
mother” as a logical expression involving predicates, quantifiers with a domain consisting of all 
people, and logical connectives. 


Solution: The statement “If a person is female and is a parent, then this person is someone’s 
mother” can be expressed as “For every person x, if person x is female and person x is a parent, 
then there exists a person y such that person x is the mother of person y.” We introduce the 
propositional functions F(x) to represent “x is female,’ P(x) to represent “x is a parent,’ and 
M(x, y) to represent “x is the mother of y.” The original statement can be represented as 


Vx((F(x) A P(x)) > 3yM(x, y)). 


Using the null quantification rule in part (b) of Exercise 49 in Section 1.4, we can move dy to 
the left so that it appears just after Vx, because y does not appear in F(x) A P(x). We obtain the 
logically equivalent expression 


Vxdy((F(x) A P(x)) > M(x, y)). < 


Express the statement “Everyone has exactly one best friend” as a logical expression involving 
predicates, quantifiers with a domain consisting of all people, and logical connectives. 


Solution: The statement “Everyone has exactly one best friend” can be expressed as “For every 
person x, person x has exactly one best friend.” Introducing the universal quantifier, we see 
that this statement is the same as “Vx(person x has exactly one best friend), where the domain 
consists of all people. 

To say that x has exactly one best friend means that there is a person y who is the best friend 
of x, and furthermore, that for every person z, if person z is not person y, then z is not the best 
friend of x. When we introduce the predicate B(x, y) to be the statement “y is the best friend of 
x,” the statement that x has exactly one best friend can be represented as 


IB, y) A Vez # y) > “BG, z))). 
Consequently, our original statement can be expressed as 
Wxdy(BO, y) A W2((z # y) > “BG, z))). 


[Note that we can write this statement as Yx3!yB(x, y), where J! is the “uniqueness quantifier” 
defined in Section 1.4.] < 


Use quantifiers to express the statement “There is a woman who has taken a flight on every 
airline in the world.” 


Solution: Let P(w, f) be “w has taken f” and Q(f a) be “f is a flight on a.” We can express the 
statement as 


Awvasf (P(w, f) A Of a)), 
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where the domains of discourse for w, f, and a consist of all the women in the world, all airplane 
flights, and all airlines, respectively. 
The statement could also be expressed as 


JwYaJfR(w, f, a), 


where R(w,f,a) is “w has taken f on a.” Although this is more compact, it somewhat 
obscures the relationships among the variables. Consequently, the first solution is usually 
preferable. < 


1.5.7 Negating Nested Quantifiers 


Statements involving nested quantifiers can be negated by successively applying the rules for 
negating statements involving a single quantifier. This is illustrated in Examples 14-16. 


Express the negation of the statement Vx3y(xy = 1) so that no negation precedes a quantifier. 


Solution: By successively applying De Morgan’s laws for quantifiers in Table 2 of 
Section 1.4, we can move the negation in ~Vxdy(xy = 1) inside all the quantifiers. We find that 
aVxdy(xy = 1) is equivalent to 4x73y(xy = 1), which is equivalent to 3xVy~(xy = 1). Because 
—=(xy = 1) can be expressed more simply as xy # 1, we conclude that our negated statement can 
be expressed as AxVy(xy # 1). < 


Use quantifiers to express the statement that “There does not exist a woman who has taken a 
flight on every airline in the world.” 


Solution: This statement is the negation of the statement “There is a woman who has taken 
a flight on every airline in the world” from Example 13. By Example 13, our statement can 
be expressed as ~3JwYaJf (P(w, f) A Q(f, a)), where P(w, f) is “w has taken f” and Q(f, a) is 
“f is a flight on a.” By successively applying De Morgan’s laws for quantifiers in Table 2 of 
Section 1.4 to move the negation inside successive quantifiers and by applying De Morgan’s 
law for negating a conjunction in the last step, we find that our statement is equivalent to each 
of this sequence of statements: 


VwrVaat(P(w, f) A OCF, a) = Vwaa75f (P(w, f) A OU, a)) 
= Vwdavf-(P(w, f) A OC, a)) 
= Vwiavf(-P(w, f) V AOC, a)). 


This last statement states “For every woman there is an airline such that for all flights, this 
woman has not taken that flight or that flight is not on this airline.” 


(Requires calculus) Use quantifiers and predicates to express the fact that lim „4 f(x) does not 
exist where f(x) is a real-valued function of a real variable x and a belongs to the domain of f 
Solution: To say that lim,_,, f(x) does not exist means that for all real numbers L, 
lim, f(x) Æ L. By using Example 8, the statement lim, f(x) # L can be expressed as 


ag xa 


~ve >0 46 >0 Vx(0 < |x- a| < 6 > |f@) -L| < e). 
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Successively applying the rules for negating quantified expressions, we construct this sequence 
of equivalent statements: 


~ve >0 46>0 Vx(0< |x — al <6 > |fŒ)-— L|<e) 
= de>0 756>0 Vx(0<|x —al<6 > |[f() -L|<e) 
= de>0 V6 >0 Ax(0< |x —al<6 > |fŒx)-— L| <e) 
= de>0 V6>0 Ax 7(0<|x —a|<6 > |f@) -L|<e) 


= de>0 Vb >0 Ax(O<|x-— al <6 A |f@) —L| 2e). 


In the last step we used the equivalence ~(p > g)=pA-gq, which follows from the fifth 
equivalence in Table 7 of Section 1.3. 
Because the statement “lim,_,,f(x) does not exist” means for all real numbers L, 


lim,.,., f(x) £ L, this can be expressed as 


VLie>0 V6>0 Ax(0 < |x- a| < 6A |f@) — LI > e). 


This last statement says that for every real number L there is a real number e > 0 such 
that for every real number 6 > 0, there exists a real number x such that 0 < |x —a| < ô and 


Fœ -L| > e. 


Exercises 


1. Translate these statements into English, where the do- 


main for each variable consists of all real numbers. 
a) Vxdy(x < y) 

b) VxVy(( 2 0) A Q 2 0) > œ 2 0) 

c) VxVydz(xy = z) 


. Translate these statements into English, where the do- 


main for each variable consists of all real numbers. 
a) IxVyQy = y) 

b) VxVy(((x > 0) A (y <0) > œ-y> 0) 

c) VWxVydz(x = y + z) 


. Let Q(x, y) be the statement “x has sent an e-mail mes- 


sage to y,’ where the domain for both x and y consists of 
all students in your class. Express each of these quantifi- 
cations in English. 


a) Ix3yQQ, y) 
c) VxAyQ(, y) 
e) VyIxQQ, y) 


b) AxVyQ, y) 
d) AyVxQ( y) 
£) YxWyQ, y) 


. Let P(x, y) be the statement “Student x has taken class y,” 


where the domain for x consists of all students in your 
class and for y consists of all computer science courses 
at your school. Express each of these quantifications in 
English. 

a) AxdyP@, y) 
c) Yx3yP(x, y) 


e) YyaxP(x, y) 


b) AxVyP(, y) 
d) AyVxP(x, y) 
f) YxYyP(x, y) 


< 


5. Let W(x, y) mean that student x has visited website y, 


where the domain for x consists of all students in your 

school and the domain for y consists of all websites. Ex- 

press each of these statements by a simple English sen- 

tence. 

a) W(Sarah Smith, www.att.com) 

b) AxW(x, www.imdb.org) 

c) AyW(José Orez, y) 

d) 3y(W(Ashok Puri, y) A W(Cindy Yoon, y)) 

e) dyVz(y 4 (David Belcher) A (W(David Belcher, z) > 
W0,z))) 

f) Axdyvz((« # y) A (Wx z) e WO, 2) 


. Let C(x, y) mean that student x is enrolled in class y, 


where the domain for x consists of all students in your 
school and the domain for y consists of all classes being 
given at your school. Express each of these statements by 
a simple English sentence. 

a) C(Randy Goldberg, CS 252) 

b) 3xC(x, Math 695) 

c) dAyC(Carol Sitea, y) 

d) 3x(C(x, Math 222) A C(x, CS 252)) 

e) Axdw2((« # y) A (Cœ z) > CO, 2))) 

£) IxAyYz(&æ # y) A (Cœ z) e CO, 3)) 


. Let T(x, y) mean that student x likes cuisine y, where the 


domain for x consists of all students at your school and 
the domain for y consists of all cuisines. Express each of 
these statements by a simple English sentence. 


a) 77(Abdallah Hussein, Japanese) 


10. 


11. 


b) 3xT(x, Korean) A VxT(x, Mexican) 
c) dy(T(Monique Arsenault, y) V 
T(Jay Johnson, y)) 
d) Yxvzay(& # 2) > (T, y) ATE y)) 
e) Axdzvy(TG, y) © T(z y)) 
£) Vxvz3yx(T œ, y) © TE y)) 


. Let Q(x, y) be the statement “Student x has been a con- 


testant on quiz show y.” Express each of these sentences 
in terms of Q(x, y), quantifiers, and logical connectives, 
where the domain for x consists of all students at your 
school and for y consists of all quiz shows on televi- 
sion. 


a) There is a student at your school who has been a con- 
testant on a television quiz show. 

b) No student at your school has ever been a contestant 
on a television quiz show. 

c) There is a student at your school who has been a con- 
testant on Jeopardy! and on Wheel of Fortune. 

d) Every television quiz show has had a student from 
your school as a contestant. 

e) Atleast two students from your school have been con- 
testants on Jeopardy!. 


. Let L(x, y) be the statement “x loves y,” where the domain 


for both x and y consists of all people in the world. Use 

quantifiers to express each of these statements. 

a) Everybody loves Jerry. 

b) Everybody loves somebody. 

c) There is somebody whom everybody loves. 

d) Nobody loves everybody. 

e) There is somebody whom Lydia does not love. 

f) There is somebody whom no one loves. 

g) There is exactly one person whom everybody loves. 

h) There are exactly two people whom Lynn loves. 

i) Everyone loves himself or herself. 

j) There is someone who loves no one besides himself 
or herself. 


Let F(x, y) be the statement “x can fool y,” where the do- 
main consists of all people in the world. Use quantifiers 
to express each of these statements. 


a) Everybody can fool Fred. 

b) Evelyn can fool everybody. 

c) Everybody can fool somebody. 

d) There is no one who can fool everybody. 

e) Everyone can be fooled by somebody. 

f) No one can fool both Fred and Jerry. 

g) Nancy can fool exactly two people. 

h) There is exactly one person whom everybody can 
fool. 

i) No one can fool himself or herself. 

j) There is someone who can fool exactly one person 
besides himself or herself. 

Let S(x) be the predicate “x is a student,” F(x) the pred- 

icate “x is a faculty member,” and A(x, y) the predicate 

“x has asked y a question,” where the domain consists of 

all people associated with your school. Use quantifiers to 

express each of these statements. 

a) Lois has asked Professor Michaels a question. 

b) Every student has asked Professor Gross a question. 
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c) Every faculty member has either asked Professor 
Miller a question or been asked a question by Pro- 
fessor Miller. 

d) Some student has not asked any faculty member a 
question. 

e) There is a faculty member who has never been asked 
a question by a student. 

f) Some student has asked every faculty member a ques- 
tion. 

g) There is a faculty member who has asked every other 
faculty member a question. 

h) Some student has never been asked a question by a 
faculty member. 


Let /(x) be the statement “x has an Internet connection” 

and C(x, y) be the statement “x and y have chatted over 

the Internet,” where the domain for the variables x and y 

consists of all students in your class. Use quantifiers to 

express each of these statements. 

a) Jerry does not have an Internet connection. 

b) Rachel has not chatted over the Internet with 
Chelsea. 

c) Jan and Sharon have never chatted over the Internet. 

d) No one in the class has chatted with Bob. 

e) Sanjay has chatted with everyone except Joseph. 

f) Someone in your class does not have an Internet con- 
nection. 

g) Not everyone in your class has an Internet connec- 
tion. 

h) Exactly one student in your class has an Internet con- 
nection. 

i) Everyone except one student in your class has an In- 
ternet connection. 

j) Everyone in your class with an Internet connection 
has chatted over the Internet with at least one other 
student in your class. 

k) Someone in your class has an Internet connection but 
has not chatted with anyone else in your class. 

1D) There are two students in your class who have not 
chatted with each other over the Internet. 

m) There is a student in your class who has chatted with 
everyone in your class over the Internet. 

n) There are at least two students in your class who have 
not chatted with the same person in your class. 

0) There are two students in the class who between them 
have chatted with everyone else in the class. 


Let M(x, y) be “x has sent y an e-mail message” and 

T(x, y) be “x has telephoned y,’ where the domain con- 

sists of all students in your class. Use quantifiers to ex- 

press each of these statements. (Assume that all e-mail 

messages that were sent are received, which is not the 

way things often work.) 

a) Chou has never sent an e-mail message to Koko. 

b) Arlene has never sent an e-mail message to or tele- 
phoned Sarah. 

c) José has never received an e-mail message from Deb- 
orah. 

d) Every student in your class has sent an e-mail mes- 
sage to Ken. 

e) No one in your class has telephoned Nina. 
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f) Everyone in your class has either telephoned Avi or 
sent him an e-mail message. 

g) There is a student in your class who has sent every- 
one else in your class an e-mail message. 

h) There is someone in your class who has either sent an 
e-mail message or telephoned everyone else in your 
class. 

i) There are two different students in your class who 
have sent each other e-mail messages. 

j) There is a student who has sent himself or herself an 
e-mail message. 

k) There is a student in your class who has not received 
an e-mail message from anyone else in the class and 
who has not been called by any other student in the 
class. 

D) Every student in the class has either received an e- 
mail message or received a telephone call from an- 
other student in the class. 

m) There are at least two students in your class such that 
one student has sent the other e-mail and the second 
student has telephoned the first student. 

n) There are two different students in your class who 
between them have sent an e-mail message to or tele- 
phoned everyone else in the class. 

Use quantifiers and predicates with more than one vari- 

able to express these statements. 


a) There is a student in this class who can speak Hindi. 

b) Every student in this class plays some sport. 

c) Some student in this class has visited Alaska but has 
not visited Hawaii. 

d) All students in this class have learned at least one pro- 
gramming language. 

e) There is a student in this class who has taken ev- 
ery course offered by one of the departments in this 
school. 

f) Some student in this class grew up in the same town 
as exactly one other student in this class. 

g) Every student in this class has chatted with at least 
one other student in at least one chat group. 

Use quantifiers and predicates with more than one vari- 

able to express these statements. 

a) Every computer science student needs a course in dis- 
crete mathematics. 

b) There is a student in this class who owns a personal 
computer. 

c) Every student in this class has taken at least one com- 
puter science course. 

d) There is a student in this class who has taken at least 
one course in computer science. 

e) Every student in this class has been in every building 
on campus. 

f) There is a student in this class who has been in every 
room of at least one building on campus. 

g) Every student in this class has been in at least one 
room of every building on campus. 
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. A discrete mathematics class contains 1 mathematics 
major who is a freshman, 12 mathematics majors who 
are sophomores, 15 computer science majors who are 
sophomores, 2 mathematics majors who are juniors, 2 
computer science majors who are juniors, and 1 computer 
science major who is a senior. Express each of these state- 
ments in terms of quantifiers and then determine its truth 
value. 


a) There is a student in the class who is a junior. 

b) Every student in the class is a computer science ma- 
jor. 

c) There is a student in the class who is neither a math- 
ematics major nor a junior. 

d) Every student in the class is either a sophomore or a 
computer science major. 

e) There is a major such that there is a student in the 
class in every year of study with that major. 


Express each of these system specifications using pred- 

icates, quantifiers, and logical connectives, if neces- 

sary. 

a) Every user has access to exactly one mailbox. 

b) There is a process that continues to run during all er- 
ror conditions only if the kernel is working correctly. 

c) All users on the campus network can access all web- 
sites whose url has a .edu extension. 

*d) There are exactly two systems that monitor every re- 

mote server. 


. Express each of these system specifications using pred- 
icates, quantifiers, and logical connectives, if neces- 
sary. 

a) At least one console must be accessible during every 
fault condition. 

b) The e-mail address of every user can be retrieved 
whenever the archive contains at least one message 
sent by every user on the system. 

c) For every security breach there is at least one mecha- 
nism that can detect that breach if and only if there is 
a process that has not been compromised. 

d) There are at least two paths connecting every two dis- 
tinct endpoints on the network. 

e) No one knows the password of every user on the sys- 
tem except for the system administrator, who knows 
all passwords. 


Express each of these statements using mathematical and 

logical operators, predicates, and quantifiers, where the 

domain consists of all integers. 

a) The sum of two negative integers is negative. 

b) The difference of two positive integers is not neces- 
sarily positive. 

c) The sum of the squares of two integers is greater than 
or equal to the square of their sum. 

d) The absolute value of the product of two integers is 
the product of their absolute values. 

Express each of these statements using predicates, quan- 

tifiers, logical connectives, and mathematical operators 

where the domain consists of all integers. 


a) The product of two negative integers is positive. 
b) The average of two positive integers is positive. 
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c) The difference of two negative integers is not neces- 
sarily negative. 

d) The absolute value of the sum of two integers does 
not exceed the sum of the absolute values of these in- 
tegers. 

Use predicates, quantifiers, logical connectives, and 

mathematical operators to express the statement that ev- 

ery positive integer is the sum of the squares of four in- 
tegers. 

Use predicates, quantifiers, logical connectives, and 

mathematical operators to express the statement that there 

is a positive integer that is not the sum of three squares. 

Express each of these mathematical statements using 

predicates, quantifiers, logical connectives, and mathe- 

matical operators. 

a) The product of two negative real numbers is positive. 

b) The difference of a real number and itself is zero. 

c) Every positive real number has exactly two square 
roots. 

d) A negative real number does not have a square root 
that is a real number. 

Translate each of these nested quantifications into an En- 

glish statement that expresses a mathematical fact. The 

domain in each case consists of all real numbers. 

a) AxVy(~+y=y) 

b) YxYx(&Œ 2 0) A O < 0)) > œ- y > 0)) 

c) JIKE < 0) A00- y > 0)) 

d) VxVy((x + 0) A O + 0) e (ay #0) 

Translate each of these nested quantifications into an En- 

glish statement that expresses a mathematical fact. The 

domain in each case consists of all real numbers. 

a) ArVy(y = y) 

b) Vx¥y((@ < 0) A O < 0)) > (ay > 0)) 

©) Axdy(@? > y) AQ <y)) 

d) VxVydz(x +y =z) 

Let Q(x, y) be the statement “x + y = x — y.” If the do- 

main for both variables consists of all integers, what are 

the truth values? 


a) Q(1, 1) 

ce) VyQd, y) 
e) IxIyQ, y) 
g) IyYxQ(x, y) 
i) VxVyQCQ, y) 
Determine the truth value of each of these statements if 
the domain for all variables consists of all integers. 

a) Ynam(n? < m) b) AnVm(n < m?) 

c) Vndm(n +m = 0) d) 3nYm(nm = m) 

e) InImn? + m? = 5) f) Inam? + m? = 6) 

g) dIndam(n+m=4An-—m=1) 

h) 3anðam(n+m=4^An-m=2) 

i) YnYm3p(p = (m + n)/2) 

Determine the truth value of each of these statements 
if the domain of each variable consists of all real num- 
bers. 

a) YxIy&? = y) 
c) AxVy(xy = 0) 


b) Q(2, 0) 

d) 3xQ(x, 2) 
f) Yx3IyQ x y) 
h) VyaxQQ, y) 


b) Vxdy(x = y’) 
d) Ixy +y fy + x) 
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e) Vx(x #0 > Iyæw = 1)) 

f) avy #0 > xy = 1) 

g) Vxdya+y= 1) 

h) ax3y(x + 2y = 2 A 2x + 4y = 5) 

i) Vxdya+y=2A2x—-y=1) 

j) YxYyaz(z = (x + y)/2) 

Suppose the domain of the propositional function P(x, y) 
consists of pairs x and y, where x is 1, 2, or 3 and yis 1, 2, 
or 3. Write out these propositions using disjunctions and 
conjunctions. 

a) VxVyP(x, y) b) AxayP@, y) 

c) AxVyP(, y) d) Yya3xP(x, y) 

Rewrite each of these statements so that negations ap- 
pear only within predicates (that is, so that no negation 
is outside a quantifier or an expression involving logical 
connectives). 

a) ~73yą3xP(x, y) 

c) =I(QO) A VarRQ, y)) 
d) 7Sy(AxR(x, y) V YxS(x, y)) 

e) aAy(VxdzT x, y, z) V AxVZU(, y, z)) 

Express the negations of each of these statements so that 
all negation symbols immediately precede predicates. 

a) VxdyVzT (x, y, z) 

b) VxdyP@, y) V Yx3yQ(x, y) 

c) Vxdy(P(x, y) A IzR(x, y, z)) 

d) Vxdy(P@ y) > QQ, y)) 

Express the negations of each of these statements so that 
all negation symbols immediately precede predicates. 

a) 3zVyYxT(x, y, z) 

b) 3x3yP(x, y) A YxYyQ(x, y) 

ce) xIx, y) + Q, xX) 

d) Yyaxaz(T(x, y, z) V Q(x, y)) 

Rewrite each of these statements so that negations ap- 
pear only within predicates (that is, so that no negation 
is outside a quantifier or an expression involving logical 
connectives). 

a) =VxYyP(x, y) 

c) AVyVx(PO, y) V Os y)) 
d) >(Axdy7P(, y) A WxVyO(% y)) 

e) ~avx(3yYzP(x, y, z) A AzVyPG, y, z)) 

Find a common domain for the variables x, y, and 
z for which the statement VxVy((x # y) > Vz((z = x) V 
(z = y))) is true and another domain for which it is false. 


b) =VxdyPQ, y) 


b) ~vyaxP(x, y) 


Find a common domain for the variables x,y,z, 
and w for which the statement VaVyVzdw((w # x) A 
(w # y) A (w Æ Z)) Is true and another common domain 
for these variables for which it is false. 


Express each of these statements using quantifiers. Then 

form the negation of the statement so that no negation is 

to the left of a quantifier. Next, express the negation in 

simple English. (Do not simply use the phrase “It is not 

the case that.”) 

a) No one has lost more than one thousand dollars play- 
ing the lottery. 

b) There is a student in this class who has chatted with 
exactly one other student. 
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c) No student in this class has sent e-mail to exactly two 
other students in this class. 

d) Some student has solved every exercise in this book. 

e) No student has solved at least one exercise in every 
section of this book. 

Express each of these statements using quantifiers. Then 

form the negation of the statement so that no negation is 

to the left of a quantifier. Next, express the negation in 

simple English. (Do not simply use the phrase “It is not 

the case that.”) 

a) Every studentin this class has taken exactly two math- 
ematics classes at this school. 

b) Someone has visited every country in the world ex- 
cept Libya. 

c) Noone has climbed every mountain in the Himalayas. 

d) Every movie actor has either been in a movie with 
Kevin Bacon or has been in a movie with someone 
who has been in a movie with Kevin Bacon. 

Express the negations of these propositions using quan- 

tifiers, and in English. 

a) Every student in this class likes mathematics. 

b) There is a student in this class who has never seen a 
computer. 

c) There is a student in this class who has taken every 
mathematics course offered at this school. 

d) There is a student in this class who has been in at least 
one room of every building on campus. 

Find a counterexample, if possible, to these universally 

quantified statements, where the domain for all variables 

consists of all integers. 

a) WixVy(x" =y >x =y) 

b) Vxdy(? = x) 

c) VxVy(xy > x) 

Find a counterexample, if possible, to these universally 

quantified statements, where the domain for all variables 

consists of all integers. 

a) Wxdy(x = 1/y) 

b) Vxdy(G" — x < 100) 

©) Vay? y) 

Use quantifiers to express the associative law for multi- 

plication of real numbers. 

Use quantifiers to express the distributive laws of multi- 

plication over addition for real numbers. 


Use quantifiers and logical connectives to express the fact 
that every linear polynomial (that is, polynomial of de- 
gree 1) with real coefficients and where the coefficient of 
x is nonzero, has exactly one real root. 

Use quantifiers and logical connectives to express the fact 
that a quadratic polynomial with real number coefficients 
has at most two real roots. 

Determine the truth value of the statement Vxdy(xy = 1) 
if the domain for the variables consists of 


a) the nonzero real numbers. 

b) the nonzero integers. 

c) the positive real numbers. 

46. Determine the truth value of the statement SxVy(x < y?) 
if the domain for the variables consists of 
a) the positive real numbers. 

b) the integers. 

c) the nonzero real numbers. 

47. Show that the two statements ~3xYyP(x, y) and 
Vxdy-P(x, y), where both quantifiers over the first vari- 
able in P(x, y) have the same domain, and both quantifiers 
over the second variable in P(x, y) have the same domain, 
are logically equivalent. 

* 48. Show that VxP(x) VVxQ(x) and VxVy(P(x) v Q)), 
where all quantifiers have the same nonempty domain, 
are logically equivalent. (The new variable y is used to 
combine the quantifications correctly.) 

* 49, a) Show that VxP(x) A 4xQ(x) is logically equivalent to 
Vxdy (P(x) A Q(y)), where all quantifiers have the 
same nonempty domain. 

b) Show that VxP(x) V IxQ(x) is equivalent to Vxdy 
(P(x) v Q(y)), where all quantifiers have the same 
nonempty domain. 

A statement is in prenex normal form (PNF) if and only if 

it is of the form 

Q 1x1 Qx +++ O,X, P(X], Xa, «++ XA) 

where each Q, i = 1, 2, ..., k, is either the existential quanti- 

fier or the universal quantifier, and P(x, ...,x,) is a predicate 

involving no quantifiers. For example, 4xVy(P(, y) A Q(y)) is 
in prenex normal form, whereas 4xP(x) V VxQ(x) is not (be- 
cause the quantifiers do not all occur first). 

Every statement formed from propositional variables, 
predicates, T, and F using logical connectives and quanti- 
fiers is equivalent to a statement in prenex normal form. 
Exercise 51 asks for a proof of this fact. 

*50. Put these statements in prenex normal form. [Hint: Use 
logical equivalence from Tables 6 and 7 in Section 1.3, 
Table 2 in Section 1.4, Example 19 in Section 1.4, 
Exercises 47 and 48 in Section 1.4, and Exercises 48 
and 49.] 

a) AxP(x) V AxQ(x) V A, where A is a proposition not in- 
volving any quantifiers 

b) 7(VxP(x) V YxQ(x)) 

ce) AxP(x) > IxQ(x) 


** 51. Show how to transform an arbitrary statement to a state- 


ment in prenex normal form that is equivalent to the given 
statement. (Note: A formal solution of this exercise re- 
quires use of structural induction, covered in Section 5.3.) 

*52. Express the quantification 4!xP(x), introduced in Sec- 
tion 1.4, using universal quantifications, existential quan- 
tifications, and logical operators. 
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Rules of Inference 


1.6.1 Introduction 


Later in this chapter we will study proofs. Proofs in mathematics are valid arguments that estab- 
lish the truth of mathematical statements. By an argument, we mean a sequence of statements 
that end with a conclusion. By valid, we mean that the conclusion, or final statement of the ar- 
gument, must follow from the truth of the preceding statements, or premises, of the argument. 
That is, an argument is valid if and only if it is impossible for all the premises to be true and 
the conclusion to be false. To deduce new statements from statements we already have, we use 
rules of inference which are templates for constructing valid arguments. Rules of inference are 
our basic tools for establishing the truth of statements. 

Before we study mathematical proofs, we will look at arguments that involve only com- 
pound propositions. We will define what it means for an argument involving compound propo- 
sitions to be valid. Then we will introduce a collection of rules of inference in propositional 
logic. These rules of inference are among the most important ingredients in producing valid 
arguments. After we illustrate how rules of inference are used to produce valid arguments, we 
will describe some common forms of incorrect reasoning, called fallacies, which lead to invalid 
arguments. 

After studying rules of inference in propositional logic, we will introduce rules of infer- 
ence for quantified statements. We will describe how these rules of inference can be used 
to produce valid arguments. These rules of inference for statements involving existential and 
universal quantifiers play an important role in proofs in computer science and mathematics, 
although they are often used without being explicitly mentioned. 

Finally, we will show how rules of inference for propositions and for quantified statements 
can be combined. These combinations of rule of inference are often used together in complicated 
arguments. 


1.6.2 Valid Arguments in Propositional Logic 


Consider the following argument involving propositions (which, by definition, is a sequence of 
propositions): 


“If you have a current password, then you can log onto the network.” 
“You have a current password.” 

Therefore, 

“You can log onto the network.” 


We would like to determine whether this is a valid argument. That is, we would like to de- 
termine whether the conclusion “You can log onto the network” must be true when the premises 
“If you have a current password, then you can log onto the network” and “You have a current 
password” are both true. 

Before we discuss the validity of this particular argument, we will look at its form. Use p 
to represent “You have a current password” and q to represent “You can log onto the network.” 
Then, the argument has the form 


pq 
P 
-q 


where .*. is the symbol that denotes “therefore.” 
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Definition 1 


We know that when p and q are propositional variables, the statement ((p > q) Ap) > qis 
a tautology (see Exercise 12(c) in Section 1.3). In particular, when both p > q and p are true, 
we know that g must also be true. We say this form of argument is valid because whenever all 
its premises (all statements in the argument other than the final one, the conclusion) are true, 
the conclusion must also be true. Now suppose that both “If you have a current password, then 
you can log onto the network” and “You have a current password” are true statements. When 
we replace p by “You have a current password” and q by “You can log onto the network,” it 
necessarily follows that the conclusion “You can log onto the network” is true. This argument 
is valid because its form is valid. Note that whenever we replace p and q by propositions where 
p > q and p are both true, then q must also be true. 

What happens when we replace p and q in this argument form by propositions where not 
both p and p > q are true? For example, suppose that p represents “You have access to the 
network” and q represents “You can change your grade” and that p is true, but p > q is false. 
The argument we obtain by substituting these values of p and q into the argument form is 


“If you have access to the network, then you can change your grade.” 
“You have access to the network.” 


.“. “You can change your grade.” 


The argument we obtained is a valid argument, but because one of the premises, namely the first 
premise, is false, we cannot conclude that the conclusion is true. (Most likely, this conclusion 
is false.) 

In our discussion, to analyze an argument, we replaced propositions by propositional vari- 
ables. This changed an argument to an argument form. We saw that the validity of an argument 
follows from the validity of the form of the argument. We summarize the terminology used to 
discuss the validity of arguments with our definition of the key notions. 


An argument in propositional logic is a sequence of propositions. All but the final proposition 
in the argument are called premises and the final proposition is called the conclusion. An 
argument is valid if the truth of all its premises implies that the conclusion is true. 

An argument form in propositional logic is a sequence of compound propositions in- 
volving propositional variables. An argument form is valid if no matter which particular 
propositions are substituted for the propositional variables in its premises, the conclusion is 
true if the premises are all true. 


Remark: From the definition of a valid argument form we see that the argument form with 
premises p,, P2» ---, Pp and conclusion q is valid exactly when (p; Ap. A+: Ap,) > g iS a 
tautology. 


The key to showing that an argument in propositional logic is valid is to show that its 
argument form is valid. Consequently, we would like techniques to show that argument forms 
are valid. We will now develop methods for accomplishing this task. 


1.6.3 Rules of Inference for Propositional Logic 


We can always use a truth table to show that an argument form is valid. We do this by showing 
that whenever the premises are true, the conclusion must also be true. However, this can be 
a tedious approach. For example, when an argument form involves 10 different propositional 
variables, to use a truth table to show this argument form is valid requires 2'° = 1024 different 
rows. Fortunately, we do not have to resort to truth tables. Instead, we can first establish the 


EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 
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validity of some relatively simple argument forms, called rules of inference. These rules of 
inference can be used as building blocks to construct more complicated valid argument forms. 
We will now introduce the most important rules of inference in propositional logic. 

The tautology (p A (p > q)) > qis the basis of the rule of inference called modus ponens, 
or the law of detachment. (Modus ponens is Latin for mode that affirms.) This tautology leads 
to the following valid argument form, which we have already seen in our initial discussion about 
arguments (where, as before, the symbol .*. denotes “therefore’’): 


P 
pq 
-g 


Using this notation, the hypotheses are written in a column, followed by a horizontal bar, fol- 
lowed by a line that begins with the therefore symbol and ends with the conclusion. In particular, 
modus ponens tells us that if a conditional statement and the hypothesis of this conditional state- 
ment are both true, then the conclusion must also be true. Example 1 illustrates the use of modus 
ponens. 


Suppose that the conditional statement “If it snows today, then we will go skiing” and its hy- 
pothesis, “It is snowing today,” are true. Then, by modus ponens, it follows that the conclusion 
of the conditional statement, “We will go skiing,” is true. < 


As we mentioned earlier, a valid argument can lead to an incorrect conclusion if one or 
more of its premises is false. We illustrate this again in Example 2. 


Determine whether the argument given here is valid and determine whether its conclusion must 
be true because of the validity of the argument. 


2 
“Tf V2 > 5, then (/2)° > G) . We know that V2 > 5. Consequently, 
2 a = ee 


Solution: Let p be the proposition «4/2 > 5” and q the proposition “2 > GP” The premises 
of the argument are p > q and p, and q is its conclusion. This argument is valid because it 
is constructed by using modus ponens, a valid argument form. However, one of its premises, 
/2 > 5, is false. Consequently, we cannot conclude that the conclusion is true. Furthermore, 


note that the conclusion of this argument is false, because 2 < 2, < 


There are many useful rules of inference for propositional logic. Perhaps the most widely 
used of these are listed in Table 1. Exercises 13-16, 25, 33, and 34 in Section 1.3 ask for the 
verifications that these rules of inference are valid argument forms. We now give examples of 
arguments that use these rules of inference. In each argument, we first use propositional variables 
to express the propositions in the argument. We then show that the resulting argument form is 
a rule of inference from Table 1. 


State which rule of inference is the basis of the following argument: “It is below freezing now. 


Therefore, it is below freezing or raining now.” 


Solution: Let p be the proposition “It is below freezing now,” and let q be the proposition “It is 
raining now.” Then this argument is of the form 


P 
“pV 
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TABLE 1 Rules of Inference. 
Rule of Inference Tautology 


o 
q^ > q4) > p Modus tollens 


(pr qgAqrr)-prr) Hypothetical syllogism 


(PV q) An7p) > 4 Disjunctive syllogism 
a 
p^q)>p Simplification 


Dn 
S 


This is an argument that uses the addition rule. < 


EXAMPLE 4 State which rule of inference is the basis of the following argument: “It is below freezing and 
raining now. Therefore, it is below freezing now.” 


Solution: Let p be the proposition “It is below freezing now,” and let g be the proposition “It is 
raining now.” This argument is of the form 


DA 
Esp 


This argument uses the simplification rule. < 


EXAMPLE 5 State which rule of inference is used in the argument: 


If it rains today, then we will not have a barbecue today. If we do not have a barbecue today, 
then we will have a barbecue tomorrow. Therefore, if it rains today, then we will have a 
barbecue tomorrow. 


EXAMPLE 6 


Extra » 
Examples 


EXAMPLE 7 
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Solution: Let p be the proposition “It is raining today,” let q be the proposition “We will not 
have a barbecue today,” and let r be the proposition “We will have a barbecue tomorrow.” Then 
this argument is of the form 


p>q 
q>r 
por 


Hence, this argument is a hypothetical syllogism. < 


1.6.4 Using Rules of Inference to Build Arguments 


When there are many premises, several rules of inference are often needed to show that an 
argument is valid. This is illustrated by Examples 6 and 7, where the steps of arguments are 
displayed on separate lines, with the reason for each step explicitly stated. These examples also 
show how arguments in English can be analyzed using rules of inference. 


Show that the premises “It is not sunny this afternoon and it is colder than yesterday,” “We will 
go swimming only if it is sunny,” “If we do not go swimming, then we will take a canoe trip,” 
and “If we take a canoe trip, then we will be home by sunset” lead to the conclusion “We will 
be home by sunset.” 


Solution: Let p be the proposition “It is sunny this afternoon,” g the proposition “It is colder 
than yesterday,” r the proposition “We will go swimming,” s the proposition “We will take a 
canoe trip,” and f the proposition “We will be home by sunset.” Then the premises become 
ap A q,r > p, ar > s, and s > t. The conclusion is simply t. We need to give a valid argument 
with premises =p A q, r > p, ar > s, and s > t and conclusion t. 

We construct an argument to show that our premises lead to the desired conclusion as fol- 
lows. 


Step Reason 

l. spAq Premise 

2. ap Simplification using (1) 
3.r>p Premise 

4r Modus tollens using (2) and (3) 
5. ar> s Premise 

6. s Modus ponens using (4) and (5) 
7.s—>t Premise 

8. t Modus ponens using (6) and (7) 


Note that we could have used a truth table to show that whenever each of the four hypotheses 
is true, the conclusion is also true. However, because we are working with five propositional 
variables, p, q, r, s, and t, such a truth table would have 32 rows. < 


Show that the premises “If you send me an e-mail message, then I will finish writing the pro- 
gram,” “If you do not send me an e-mail message, then I will go to sleep early,” and “If I go 
to sleep early, then I will wake up feeling refreshed” lead to the conclusion “If I do not finish 
writing the program, then I will wake up feeling refreshed.” 


Solution: Let p be the proposition “You send me an e-mail message,” q the proposition “I will 
finish writing the program,” r the proposition “I will go to sleep early,” and s the proposition “I 


78 


1 / The Foundations: Logic and Proofs 


Links » 


EXAMPLE 8 


Extra > 
Examples 


EXAMPLE 9 


will wake up feeling refreshed.” Then the premises are p > q, =p > r, and r > s. The desired 
conclusion is =q — s. We need to give a valid argument with premises p > q, ap > r, and 
r > sand conclusion 7g > s. 

This argument form shows that the premises lead to the desired conclusion. 


Step Reason 

l.poqd Premise 

2. 7q > =p Contrapositive of (1) 

3. ap >r Premise 

4. ~q >r Hypothetical syllogism using (2) and (3) 

5. r>s Premise 

6. 7q >s Hypothetical syllogism using (4) and (5) 4 


1.6.5 Resolution 


Computer programs have been developed to automate the task of reasoning and proving theo- 
rems. Many of these programs make use of a rule of inference known as resolution. This rule 
of inference is based on the tautology 


(pVqg)A(7pVr)) > (Vr). 


(Exercise 34 in Section 1.3 asks for the verification that this is a tautology.) The final disjunction 
in the resolution rule, q V r, is called the resolvent. When we let q = r in this tautology, we 
obtain (p V q) A (=p V q) > q. Furthermore, when we let r = F, we obtain (p V q) A (p) > q 
(because q V F = q), which is the tautology on which the rule of disjunctive syllogism is based. 


Use resolution to show that the hypotheses “Jasmine is skiing or it is not snowing” and “It is 
snowing or Bart is playing hockey” imply that “Jasmine is skiing or Bart is playing hockey.” 


Solution: Let p be the proposition “It is snowing,” q the proposition “Jasmine is skiing,” and 
r the proposition “Bart is playing hockey.” We can represent the hypotheses as =p V q and p V 
r, respectively. Using resolution, the proposition g V r, “Jasmine is skiing or Bart is playing 
hockey,” follows. < 


Resolution plays an important role in programming languages based on the rules of logic, 
such as Prolog (where resolution rules for quantified statements are applied). Furthermore, it 
can be used to build automatic theorem proving systems. To construct proofs in propositional 
logic using resolution as the only rule of inference, the hypotheses and the conclusion must be 
expressed as clauses, where a clause is a disjunction of variables or negations of these variables. 
We can replace a statement in propositional logic that is not a clause by one or more equivalent 
statements that are clauses. For example, suppose we have a statement of the form p V (q A r). 
Because p V (q A r) = (p V q) A (p V r), we can replace the single statement p V (q A r) by two 
statements p V q and p V r, each of which is a clause. We can replace a statement of the form 
~(p V q) by the two statements =p and ~q because De Morgan’s law tells us that ~(p V q) = 
ap A ~q. We can also replace a conditional statement p > q with the equivalent disjunction 


ap Vq. 
Show that the premises (p A q) V r and r > s imply the conclusion p V s. 
Solution: We can rewrite the premises (p A q) V r as two clauses, p V r and q V r. We can also 


replace r > s by the equivalent clause =r V s. Using the two clauses p V r and ^r V s, we can 
use resolution to conclude p V s. < 


Links > 


EXAMPLE 10 


EXAMPLE 11 
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1.6.6 Fallacies 


Several common fallacies arise in incorrect arguments. These fallacies resemble rules of infer- 
ence, but are based on contingencies rather than tautologies. These are discussed here to show 
the distinction between correct and incorrect reasoning. 

The proposition ((p > q) Aq) > pis nota tautology, because it is false when p is false and 
q is true. However, there are many incorrect arguments that treat this as a tautology. In other 
words, they treat the argument with premises p > q and q and conclusion p as a valid argument 
form, which it is not. This type of incorrect reasoning is called the fallacy of affirming the 
conclusion. 


Is the following argument valid? 


If you do every problem in this book, then you will learn discrete mathematics. You learned 
discrete mathematics. 


Therefore, you did every problem in this book. 


Solution: Let p be the proposition “You did every problem in this book.” Let q be the proposition 
“You learned discrete mathematics.” Then this argument is of the form: if p > q and q, then 
p. This is an example of an incorrect argument using the fallacy of affirming the conclusion. 
Indeed, it is possible for you to learn discrete mathematics in some way other than by doing every 
problem in this book. (You may learn discrete mathematics by reading, listening to lectures, 
doing some, but not all, the problems in this book, and so on.) < 


The proposition ((p > q) A œp) > ~q is not a tautology, because it is false when p is false 
and q is true. Many incorrect arguments use this incorrectly as a rule of inference. This type of 
incorrect reasoning is called the fallacy of denying the hypothesis. 


Let p and q be as in Example 10. If the conditional statement p > q is true, and =p is true, is it 
correct to conclude that =q is true? In other words, is it correct to assume that you did not learn 
discrete mathematics if you did not do every problem in the book, assuming that if you do every 
problem in this book, then you will learn discrete mathematics? 


Solution: It is possible that you learned discrete mathematics even if you did not do every prob- 
lem in this book. This incorrect argument is of the form p > q and =p imply =q, which is an 
example of the fallacy of denying the hypothesis. < 


1.6.7 Rules of Inference for Quantified Statements 


We have discussed rules of inference for propositions. We will now describe some important 
rules of inference for statements involving quantifiers. These rules of inference are used exten- 
sively in mathematical arguments, often without being explicitly mentioned. 

Universal instantiation is the rule of inference used to conclude that P(c) is true, where c 
is a particular member of the domain, given the premise VxP(x). Universal instantiation is used 
when we conclude from the statement “All women are wise” that “Lisa is wise,” where Lisa is 
a member of the domain of all women. 

Universal generalization is the rule of inference that states that VxP(x) is true, given the 
premise that P(c) is true for all elements c in the domain. Universal generalization is used when 
we show that VxP(x) is true by taking an arbitrary element c from the domain and showing that 
P(c) is true. The element c that we select must be an arbitrary, and not a specific, element of 
the domain. That is, when we assert from VxP(x) the existence of an element c in the domain, 
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TABLE 2 Rules of Inference for Quantified Statements. 


ae) Universal instantiation 
“. P(c) 


P(c) for an arbitrary c 


-VPO Universal generalization 
AxP(x) 


: Existential instantiation 
-. P(c) for some element c 


P(c) for some element c 
ce dxP(x) 


Existential generalization 


we have no control over c and cannot make any other assumptions about c other than it comes 
from the domain. Universal generalization is used implicitly in many proofs in mathematics and 
is seldom mentioned explicitly. However, the error of adding unwarranted assumptions about 
the arbitrary element c when universal generalization is used is all too common in incorrect 
reasoning. 

Existential instantiation is the rule that allows us to conclude that there is an element c in 
the domain for which P(c) is true if we know that 3xP(x) is true. We cannot select an arbitrary 
value of c here, but rather it must be a c for which P(c) is true. Usually we have no knowledge 
of what c is, only that it exists. Because it exists, we may give it a name (c) and continue our 
argument. 

Existential generalization is the rule of inference that is used to conclude that 4xP(x) is 
true when a particular element c with P(c) true is known. That is, if we know one element c in 
the domain for which P(c) is true, then we know that 3xP(x) is true. 

We summarize these rules of inference in Table 2. We will illustrate how some of these 
rules of inference for quantified statements are used in Examples 12 and 13. 


Show that the premises “Everyone in this discrete mathematics class has taken a course in 
computer science” and “Marla is a student in this class” imply the conclusion “Marla has taken 
a course in computer science.” 


Solution: Let D(x) denote “x is in this discrete mathematics class,” and let C(x) denote “x has 
taken a course in computer science.” Then the premises are Vx(D(x) > C(x)) and D(Marla). 
The conclusion is C(Marla). 

The following steps can be used to establish the conclusion from the premises. 


Step Reason 
1. Vx(D(x) > C(x)) Premise 
2. D(Marla) > C(Marla) Universal instantiation from (1) 
3. D(Marla) Premise 
4. C(Marla) Modus ponens from (2) and (3) 


EXAMPLE 13 Show that the premises “A student in this class has not read the book,” and “Everyone in this 


class passed the first exam” imply the conclusion “Someone who passed the first exam has not 
read the book.” 


EXAMPLE 14 
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Solution: Let C(x) be “x is in this class,” B(x) be “x has read the book,” and P(x) be “x passed 
the first exam.” The premises are 4x(C(x) A =B(x)) and Vx(C(x) > P(x)). The conclusion is 
dx(P(x) A =B(x)). These steps can be used to establish the conclusion from the premises. 


Step Reason 

1. Ax(C(x) A AB(x)) Premise 

2. C(a) A 7=B(a) Existential instantiation from (1) 
3. C(a) Simplification from (2) 

4. Vx(C(x) > P(x)) Premise 

5. C(a) > P(a) Universal instantiation from (4) 

6. P(a) Modus ponens from (3) and (5) 

7. =B(a) Simplification from (2) 

8. P(a) A =B(a) Conjunction from (6) and (7) 

9. Ax(P(x) A 7B(x)) Existential generalization from (8) 


1.6.8 Combining Rules of Inference for Propositions 
and Quantified Statements 


We have developed rules of inference both for propositions and for quantified statements. Note 
that in our arguments in Examples 12 and 13 we used both universal instantiation, a rule of in- 
ference for quantified statements, and modus ponens, a rule of inference for propositional logic. 
We will often need to use this combination of rules of inference. Because universal instantia- 
tion and modus ponens are used so often together, this combination of rules is sometimes called 
universal modus ponens. This rule tells us that if Vx(P(x) > Q(x)) is true, and if P(a) is true 
for a particular element a in the domain of the universal quantifier, then Q(a) must also be true. 
To see this, note that by universal instantiation, P(a) > Q(a) is true. Then, by modus ponens, 
Q(a) must also be true. We can describe universal modus ponens as follows: 


Vx(P(x) > Q(x)) 
P(a), where ais a particular element in the domain 


“. O(a) 


Universal modus ponens is commonly used in mathematical arguments. This is illustrated 
in Example 14. 


Assume that “For all positive integers n, if n is greater than 4, then n? is less than 2”” is true. 
Use universal modus ponens to show that 100? < 2100, 


Solution: Let P(n) denote “n > 4” and Q(n) denote “n? < 2”.” The statement “For all positive 
integers n, if n is greater than 4, then n? is less than 2”” can be represented by Vn(P(n) > Q(n)), 
where the domain consists of all positive integers. We are assuming that Vn(P(n) > Q(n)) is 
true. Note that P(100) is true because 100 > 4. It follows by universal modus ponens that Q(100) 
is true, namely, that 1007 < 2!. < 


Another useful combination of a rule of inference from propositional logic and a rule of in- 
ference for quantified statements is universal modus tollens. Universal modus tollens combines 
universal instantiation and modus tollens and can be expressed in the following way: 


Vx(P(x) > Q(x) 
=Q(a), where ais a particular element in the domain 


<. 1P(a) 
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The verification of universal modus tollens is left as Exercise 25. Exercises 26—29 develop 
additional combinations of rules of inference in propositional logic and quantified statements. 


Exercises 


1. Find the argument form for the following argument and 


determine whether it is valid. Can we conclude that the 
conclusion is true if the premises are true? 


If Socrates is human, then Socrates is mortal. 
Socrates is human. 


.. Socrates is mortal. 


. Find the argument form for the following argument and 
determine whether it is valid. Can we conclude that the 
conclusion is true if the premises are true? 


If George does not have eight legs, then he is not a 
spider. 
George is a spider. 


.. George has eight legs. 


3. What rule of inference is used in each of these argu- 


ments? 

a) Alice is a mathematics major. Therefore, Alice is ei- 
ther a mathematics major or a computer science ma- 
jor. 

b) Jerry is a mathematics major and a computer science 
major. Therefore, Jerry is a mathematics major. 

c) If it is rainy, then the pool will be closed. It is rainy. 
Therefore, the pool is closed. 

d) If it snows today, the university will close. The uni- 
versity is not closed today. Therefore, it did not snow 
today. 

e) IfI go swimming, then I will stay in the sun too long. 
If I stay in the sun too long, then I will sunburn. There- 
fore, if I go swimming, then I will sunburn. 


4. What rule of inference is used in each of these argu- 


ments? 


a) Kangaroos live in Australia and are marsupials. 
Therefore, kangaroos are marsupials. 

b) It is either hotter than 100 degrees today or the pollu- 
tion is dangerous. It is less than 100 degrees outside 
today. Therefore, the pollution is dangerous. 

c) Linda is an excellent swimmer. If Linda is an ex- 
cellent swimmer, then she can work as a lifeguard. 
Therefore, Linda can work as a lifeguard. 

d) Steve will work at a computer company this summer. 
Therefore, this summer Steve will work at a computer 
company or he will be a beach bum. 

e) If I work all night on this homework, then I can an- 
swer all the exercises. If I answer all the exercises, I 
will understand the material. Therefore, if I work all 
night on this homework, then I will understand the 
material. 


10. 


. Use rules of inference to show that the hypotheses 


“Randy works hard,” “If Randy works hard, then he is 
a dull boy,” and “If Randy is a dull boy, then he will not 
get the job” imply the conclusion “Randy will not get the 
job.” 


. Use rules of inference to show that the hypotheses “If it 


does not rain or if it is not foggy, then the sailing race will 
be held and the lifesaving demonstration will go on,” “If 
the sailing race is held, then the trophy will be awarded,” 
and “The trophy was not awarded” imply the conclusion 
“Tt rained.” 


. What rules of inference are used in this famous argu- 


ment? “All men are mortal. Socrates is a man. Therefore, 
Socrates is mortal.” 


. What rules of inference are used in this argument? “No 


man is an island. Manhattan is an island. Therefore, Man- 
hattan is not a man.” 


. For each of these collections of premises, what relevant 


conclusion or conclusions can be drawn? Explain the 
rules of inference used to obtain each conclusion from 
the premises. 


a) “If I take the day off, it either rains or snows.” “I took 
Tuesday off or I took Thursday off.” “It was sunny on 
Tuesday.” “It did not snow on Thursday.” 

b) “If I eat spicy foods, then I have strange dreams.” “I 
have strange dreams if there is thunder while I sleep.” 
“T did not have strange dreams.” 

c) “I am either clever or lucky.” “I am not lucky.” “If I 
am lucky, then I will win the lottery.” 

d) “Every computer science major has a personal com- 
puter.” “Ralph does not have a personal computer.” 
“Ann has a personal computer.” 

e) “What is good for corporations is good for the United 
States.” “What is good for the United States is good 
for you.” “What is good for corporations is for you to 
buy lots of stuff.” 

f) “All rodents gnaw their food.” “Mice are rodents.” 
“Rabbits do not gnaw their food.” “Bats are not ro- 
dents.” 


For each of these sets of premises, what relevant conclu- 
sion or conclusions can be drawn? Explain the rules of in- 
ference used to obtain each conclusion from the premises. 


a) “If I play hockey, then I am sore the next day.” “T 
use the whirlpool if I am sore.” “I did not use the 
whirlpool.” 

b) “If I work, it is either sunny or partly sunny.” “T 
worked last Monday or I worked last Friday.” “It was 
not sunny on Tuesday.” “It was not partly sunny on 
Friday.” 

c) “All insects have six legs.” “Dragonflies are insects.” 
“Spiders do not have six legs.” “Spiders eat dragon- 
flies.” 


11. 


12. 


13. 


14. 


d) “Every student has an Internet account.” “Homer 
does not have an Internet account.” “Maggie has an 
Internet account.” 

e) “All foods that are healthy to eat do not taste good.” 
“Tofu is healthy to eat.” “You only eat what tastes 
good.” “You do not eat tofu.” “Cheeseburgers are not 
healthy to eat.” 

f) “I am either dreaming or hallucinating.” “I am not 
dreaming.” “If Iam hallucinating, I see elephants run- 
ning down the road.” 

Show that the argument form with premises p}, Pz, .-., Pn 

and conclusion q > r is valid if the argument form with 

premises pj, P2, -.., Pw q, and conclusion r is valid. 


Show that the argument form with premises (p A t) > 
(rV s),q > (u At), u > p, and 7s and conclusion q > r 
is valid by first using Exercise 11 and then using rules of 
inference from Table 1. 


For each of these arguments, explain which rules of in- 
ference are used for each step. 


a) “Doug, a student in this class, knows how to write 
programs in JAVA. Everyone who knows how to 
write programs in JAVA can get a high-paying job. 
Therefore, someone in this class can get a high-paying 
job.” 

b) “Somebody in this class enjoys whale watching. Ev- 
ery person who enjoys whale watching cares about 
ocean pollution. Therefore, there is a person in this 
class who cares about ocean pollution.” 

c) “Each of the 93 students in this class owns a personal 
computer. Everyone who owns a personal computer 
can use a word processing program. Therefore, Zeke, 
a student in this class, can use a word processing pro- 
gram.” 

d) “Everyone in New Jersey lives within 50 miles of the 
ocean. Someone in New Jersey has never seen the 
ocean. Therefore, someone who lives within 50 miles 
of the ocean has never seen the ocean.” 


For each of these arguments, explain which rules of in- 
ference are used for each step. 


a) “Linda, a student in this class, owns a red convertible. 
Everyone who owns a red convertible has gotten at 
least one speeding ticket. Therefore, someone in this 
class has gotten a speeding ticket.” 

b) “Each of five roommates, Melissa, Aaron, Ralph, Ve- 
neesha, and Keeshawn, has taken a course in discrete 
mathematics. Every student who has taken a course in 
discrete mathematics can take a course in algorithms. 
Therefore, all five roommates can take a course in al- 
gorithms next year.” 

c) “All movies produced by John Sayles are wonder- 
ful. John Sayles produced a movie about coal min- 
ers. Therefore, there is a wonderful movie about coal 
miners.” 

d) “There is someone in this class who has been to 
France. Everyone who goes to France visits the 
Louvre. Therefore, someone in this class has visited 
the Louvre.” 


15. 


16. 


17. 


18. 
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For each of these arguments determine whether the argu- 
ment is correct or incorrect and explain why. 


a) All students in this class understand logic. Xavier is 
a student in this class. Therefore, Xavier understands 
logic. 

b) Every computer science major takes discrete math- 
ematics. Natasha is taking discrete mathematics. 
Therefore, Natasha is a computer science major. 

c) All parrots like fruit. My pet bird is not a parrot. 
Therefore, my pet bird does not like fruit. 

d) Everyone who eats granola every day is healthy. 
Linda is not healthy. Therefore, Linda does not eat 
granola every day. 

For each of these arguments determine whether the argu- 

ment is correct or incorrect and explain why. 


a) Everyone enrolled in the university has lived in a dor- 
mitory. Mia has never lived in a dormitory. Therefore, 
Mia is not enrolled in the university. 

b) A convertible car is fun to drive. Isaac’s car is not a 
convertible. Therefore, Isaac’s car is not fun to drive. 

c) Quincy likes all action movies. Quincy likes the 
movie Eight Men Out. Therefore, Eight Men Out is 
an action movie. 

d) Alllobstermen set at least a dozen traps. Hamilton is a 
lobsterman. Therefore, Hamilton sets at least a dozen 
traps. 

What is wrong with this argument? Let H(x) be “x is 

happy.” Given the premise 4xH(x), we conclude that 

H (Lola). Therefore, Lola is happy. 


What is wrong with this argument? Let S(x, y) be “x is 
shorter than y.” Given the premise 4sS(s, Max), it fol- 
lows that S(Max, Max). Then by existential generaliza- 
tion it follows that 4xS(x, x), so that someone is shorter 
than himself. 


Determine whether each of these arguments is valid. If an 
argument is correct, what rule of inference is being used? 
If it is not, what logical error occurs? 


a) If n is a real number such that n > 1, then n? > 1. 
Suppose that n? > 1. Then n > 1. 

b) If n is a real number with n> 3, then n? > 9. 
Suppose that n? < 9. Then n < 3. 

c) If n is a real number with n> 2, then n? > 4. 
Suppose that n < 2. Then n? < 4. 

Determine whether these are valid arguments. 

a) If xis a positive real number, then x? is a positive real 
number. Therefore, if a? is positive, where a is a real 
number, then a is a positive real number. 

b) If x* £ 0, where x is a real number, then x ¥ 0. Let a 
be a real number with a? 4 0; then a + 0. 

Which rules of inference are used to establish the 

conclusion of Lewis Carroll’s argument described in 

Example 26 of Section 1.4? 


Which rules of inference are used to establish the 
conclusion of Lewis Carroll’s argument described in 
Example 27 of Section 1.4? 
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Identify the error or errors in this argument that sup- 
posedly shows that if 3xP(x) ^A 3xQ(x) is true then 
Ax(P(x) A Q(x)) is true. 


1. AxP(x) v AxO(x) Premise 

2. AxP(x) Simplification from (1) 

3. P(c) Existential instantiation from (2) 
4. IxQ(x) Simplification from (1) 

5. Q(c) Existential instantiation from (4) 
6. P(c) A O(c) Conjunction from (3) and (5) 

7. Ax(P(x) A Q(x)) Existential generalization 


Identify the error or errors in this argument that sup- 
posedly shows that if Vx(P(x) V Q(x)) is true then 
VxP(x) V VxQ(x) is true. 


1. Vx(P(x) v Q(x)) Premise 

2. P(c) V O(c) Universal instantiation from (1) 
3. P(c) Simplification from (2) 

4. VxP(x) Universal generalization from (3) 
5. Q(c) Simplification from (2) 

6. VxQ(x) Universal generalization from (5) 
7. Wx(P(x) V VxQ(x)) Conjunction from (4) and (6) 


Justify the rule of universal modus tollens by showing 
that the premises Vx(P(x) > Q(x)) and 7Q(a) for a par- 
ticular element a in the domain, imply —P(a). 

Justify the rule of universal transitivity, which states 
that if Vx(P(x) > Q(x)) and Vx(Q(x) > R(x)) are true, 
then Vx(P(x) => R(x)) is true, where the domains of all 
quantifiers are the same. 

Use rules of inference to show that if Vx(P(x) > (Q(x) A 
S(x))) and Vx(P(x) A R(x)) are true, then Vx(R(x) A S(x)) 
is true. 

Use rules of inference to show that if Vx(P(x) V Q(x)) and 
Vx((=P(x) A Q(x)) > R(x)) are true, then Vx(AR(x) > 
P(x)) is also true, where the domains of all quantifiers 
are the same. 

Use rules of inference to show that if Vx(P(x) V Q(x)), 
Vx(7Q(x) V S(x)), Vx(R(X) > aS(x)), and Ax-P(x) are 
true, then 4x7R(x) is true. 

Use resolution to show the hypotheses “Allen is a bad 
boy or Hillary is a good girl” and “Allen is a good boy or 
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David is happy” imply the conclusion “Hillary is a good 
girl or David is happy.” 


Use resolution to show that the hypotheses “It is not rain- 
ing or Yvette has her umbrella,” “Yvette does not have 
her umbrella or she does not get wet,” and “It is raining 
or Yvette does not get wet” imply that “Yvette does not 
get wet.” 


Show that the equivalence p A =p = F can be derived us- 
ing resolution together with the fact that a conditional 
statement with a false hypothesis is true. [Hint: Let g = 
r = F in resolution. | 


Use resolution to show that the compound proposition 
(PV gq) ACpVq)A (PV 7g) A (Ap V ~q) is not satisfi- 
able. 
The Logic Problem, taken from WFF’N PROOF, The 
Game of Logic, has these two assumptions: 
1. “Logic is difficult or not many students like logic.” 
2. “If mathematics is easy, then logic is not difficult.” 
By translating these assumptions into statements involv- 
ing propositional variables and logical connectives, de- 
termine whether each of the following are valid conclu- 
sions of these assumptions: 
a) That mathematics is not easy, if many students like 
logic. 
b) That not many students like logic, if mathematics is 
not easy. 
c) That mathematics is not easy or logic is difficult. 
d) That logic is not difficult or mathematics is not easy. 
e) That if not many students like logic, then either math- 
ematics is not easy or logic is not difficult. 
Determine whether this argument, taken from Kalish and 
Montague [KaMo64], is valid. 
If Superman were able and willing to prevent evil, 
he would do so. If Superman were unable to pre- 
vent evil, he would be impotent; if he were unwilling 
to prevent evil, he would be malevolent. Superman 
does not prevent evil. If Superman exists, he is nei- 
ther impotent nor malevolent. Therefore, Superman 
does not exist. 


1.7.1 Introduction 


In this section we introduce the notion of a proof and describe methods for constructing proofs. 
A proof is a valid argument that establishes the truth of a mathematical statement. A proof can 
use the hypotheses of the theorem, if any, axioms assumed to be true, and previously proven 
theorems. Using these ingredients and rules of inference, the final step of the proof establishes 
the truth of the statement being proved. 

In our discussion we move from formal proofs of theorems toward more informal proofs. 
The arguments we introduced in Section 1.6 to show that statements involving propositions 
and quantified statements are true were formal proofs, where all steps were supplied, and the 
rules for each step in the argument were given. However, formal proofs of useful theorems can 
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be extremely long and hard to follow. In practice, the proofs of theorems designed for human 
consumption are almost always informal proofs, where more than one rule of inference may 
be used in each step, where steps may be skipped, where the axioms being assumed and the 
rules of inference used are not explicitly stated. Informal proofs can often explain to humans 
why theorems are true, while computers are perfectly happy producing formal proofs using 
automated reasoning systems. 

The methods of proof discussed in this chapter are important not only because they are used 
to prove mathematical theorems, but also for their many applications to computer science. These 
applications include verifying that computer programs are correct, establishing that operating 
systems are secure, making inferences in artificial intelligence, showing that system specifica- 
tions are consistent, and so on. Consequently, understanding the techniques used in proofs is 
essential both in mathematics and in computer science. 


1.7.2 Some Terminology 


Formally, a theorem is a statement that can be shown to be true. In mathematical writing, the 
term theorem is usually reserved for a statement that is considered at least somewhat important. 
Less important theorems sometimes are called propositions. (Theorems can also be referred to 
as facts or results.) A theorem may be the universal quantification of a conditional statement 
with one or more premises and a conclusion. However, it may be some other type of logical 
statement, as the examples later in this chapter will show. We demonstrate that a theorem is true 
with a proof. A proof is a valid argument that establishes the truth of a theorem. The statements 
used in a proof can include axioms (or postulates), which are statements we assume to be true 
(for example, the axioms for the real numbers, given in Appendix 1, and the axioms of plane 
geometry), the premises, if any, of the theorem, and previously proven theorems. Axioms may 
be stated using primitive terms that do not require definition, but all other terms used in theorems 
and their proofs must be defined. Rules of inference, together with definitions of terms, are used 
to draw conclusions from other assertions, tying together the steps of a proof. In practice, the 
final step of a proof is usually just the conclusion of the theorem. However, for clarity, we will 
often recap the statement of the theorem as the final step of a proof. 

A less important theorem that is helpful in the proof of other results is called a lemma 
(plural lemmas or lemmata). Complicated proofs are usually easier to understand when they are 
proved using a series of lemmas, where each lemma is proved individually. A corollary is a 
theorem that can be established directly from a theorem that has been proved. A conjecture is 
a statement that is being proposed to be a true statement, usually on the basis of some partial 
evidence, a heuristic argument, or the intuition of an expert. When a proof of a conjecture is 
found, the conjecture becomes a theorem. Many times conjectures are shown to be false, so they 
are not theorems. 


1.7.3 Understanding How Theorems Are Stated 


Before we introduce methods for proving theorems, we need to understand how many mathe- 
matical theorems are stated. Many theorems assert that a property holds for all elements in a 
domain, such as the integers or the real numbers. Although the precise statement of such theo- 
rems needs to include a universal quantifier, the standard convention in mathematics is to omit 
it. For example, the statement 


“If x > y, where x and y are positive real numbers, then x? > y?” 


really means 


” 


“For all positive real numbers x and y, if x > y, then x? > y?. 
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Furthermore, when theorems of this type are proved, the first step of the proof usually involves 
selecting a general element of the domain. Subsequent steps show that this element has the 
property in question. Finally, universal generalization implies that the theorem holds for all 
members of the domain. 


1.7.4 Methods of Proving Theorems 


Proving mathematical theorems can be difficult. To construct proofs we need all available am- 
munition, including a powerful battery of different proof methods. These methods provide the 
overall approach and strategy of proofs. Understanding these methods is a key component of 
learning how to read and construct mathematical proofs. Once we have chosen a proof method, 
we use axioms, definitions of terms, previously proved results, and rules of inference to com- 
plete the proof. Note that in this book we will always assume the axioms for real numbers found 
in Appendix 1. We will also assume the usual axioms whenever we prove a result about ge- 
ometry. When you construct your own proofs, be careful not to use anything but these axioms, 
definitions, and previously proved results as facts! 

To prove a theorem of the form Vx(P(x) > Q(x)), our goal is to show that P(c) > Q(c) is 
true, where c is an arbitrary element of the domain, and then apply universal generalization. In 
this proof, we need to show that a conditional statement is true. Because of this, we now focus 
on methods that show that conditional statements are true. Recall that p — q is true unless p is 
true but q is false. Note that to prove the statement p > q, we need only show that q is true if p 
is true. The following discussion will give the most common techniques for proving conditional 
statements. Later we will discuss methods for proving other types of statements. In this section, 
and in Section 1.8, we will develop a large arsenal of proof techniques that can be used to prove 
a wide variety of theorems. 

When you read proofs, you will often find the words “obviously” or “clearly.” These words 
indicate that steps have been omitted that the author expects the reader to be able to fill in. 
Unfortunately, this assumption is often not warranted and readers are not at all sure how to fill 
in the gaps. We will assiduously try to avoid using these words and try not to omit too many 
steps. However, if we included all steps in proofs, our proofs would often be excruciatingly long. 


1.7.5 Direct Proofs 


A direct proof of a conditional statement p > q is constructed when the first step is the as- 
sumption that p is true; subsequent steps are constructed using rules of inference, with the final 
step showing that q must also be true. A direct proof shows that a conditional statement p > q 
is true by showing that if p is true, then g must also be true, so that the combination p true and 
q false never occurs. In a direct proof, we assume that p is true and use axioms, definitions, and 
previously proven theorems, together with rules of inference, to show that q must also be true. 
You will find that direct proofs of many results are quite straightforward. Starting with the hy- 
pothesis and leading to the conclusion, the way forward is essentially dictated by the premises 
available at that step. However, direct proofs sometimes require particular insights and can be 
quite tricky. The first direct proofs we present here are quite straightforward; later in the text 
you will see some that require some insight. 

We will provide examples of several different direct proofs. Before we give the first example, 
we need to define some terminology. 


The integer n is even if there exists an integer k such that n = 2k, and n is odd if there exists an 
integer k such that n = 2k + 1. (Note that every integer is either even or odd, and no integer 
is both even and odd.) Two integers have the same parity when both are even or both are odd; 
they have opposite parity when one is even and the other is odd. 
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Give a direct proof of the theorem “If n is an odd integer, then n? is odd.” 


Solution: Note that this theorem states VnP((n) > Q(n)), where P(n) is “n is an odd integer’ and 
Q(n) is “n is odd.” As we have said, we will follow the usual convention in mathematical proofs 
by showing that P(n) implies Q(n), and not explicitly using universal instantiation. To begin a 
direct proof of this theorem, we assume that the hypothesis of this conditional statement is true, 
namely, we assume that n is odd. By the definition of an odd integer, it follows that n = 2k + 1, 
where k is some integer. We want to show that n? is also odd. We can square both sides of the 
equation n = 2k + 1 to obtain a new equation that expresses n?. When we do this, we find that 
n? = (2k + 1)? = 4k? + 4k + 1 = 2(2k* + 2k) + 1. By the definition of an odd integer, we can 
conclude that n? is an odd integer (it is one more than twice an integer). Consequently, we have 
proved that if n is an odd integer, then n? is an odd integer. < 


Give a direct proof that if m and n are both perfect squares, then nm is also a perfect square. (An 
integer a is a perfect square if there is an integer b such that a = b’.) 


Solution: To produce a direct proof of this theorem, we assume that the hypothesis of this con- 
ditional statement is true, namely, we assume that m and n are both perfect squares. By the 
definition of a perfect square, it follows that there are integers s and t such that m = s? and 
n = tf’. The goal of the proof is to show that mn must also be a perfect square when m and n are; 
looking ahead we see how we can show this by substituting s? for m and £? for n into mn. This 
tells us that mn = s???. Hence, mn = s*t? = (ss)(tt) = (st)(st) = (st)”, using commutativity and 
associativity of multiplication. By the definition of perfect square, it follows that mn is also a 
perfect square, because it is the square of st, which is an integer. We have proved that if m and n 
are both perfect squares, then mn is also a perfect square. < 


1.7.6 Proof by Contraposition 


Direct proofs lead from the premises of a theorem to the conclusion. They begin with the 
premises, continue with a sequence of deductions, and end with the conclusion. However, we 
will see that attempts at direct proofs often reach dead ends. We need other methods of prov- 
ing theorems of the form Vx(P(x) => Q(x)). Proofs of theorems of this type that are not direct 
proofs, that is, that do not start with the premises and end with the conclusion, are called indirect 
proofs. 

An extremely useful type of indirect proof is known as proof by contraposition. Proofs 
by contraposition make use of the fact that the conditional statement p > q is equivalent to its 
contrapositive, =q > =p. This means that the conditional statement p > q can be proved by 
showing that its contrapositive, =q — =p, is true. In a proof by contraposition of p > q, we 
take ~q as a premise, and using axioms, definitions, and previously proven theorems, together 
with rules of inference, we show that =p must follow. We will illustrate proof by contraposition 
with two examples. These examples show that proof by contraposition can succeed when we 
cannot easily find a direct proof. 


Prove that if n is an integer and 3n + 2 is odd, then n is odd. 


Solution: We first attempt a direct proof. To construct a direct proof, we first assume that 3n + 2 
is an odd integer. From the definition of an odd integer, we know that 3n + 2 = 2k + 1 for some 
integer k. Can we use this fact to show that n is odd? We see that 3n + 1 = 2k, but there does 
not seem to be any direct way to conclude that n is odd. Because our attempt at a direct proof 
failed, we next try a proof by contraposition. 


88 


1 / The Foundations: Logic and Proofs 


EXAMPLE 4 


EXAMPLE 5 


EXAMPLE 6 


The first step in a proof by contraposition is to assume that the conclusion of the conditional 
statement “If 3n + 2 is odd, then n is odd” is false; namely, assume that n is even. Then, by the 
definition of an even integer, n = 2k for some integer k. Substituting 2k for n, we find that 37n + 
2 = 3(2k) +2 = 6k +2 =2(3k + 1). This tells us that 3n + 2 is even (because it is a multiple 
of 2), and therefore not odd. This is the negation of the premise of the theorem. Because the 
negation of the conclusion of the conditional statement implies that the hypothesis is false, the 
original conditional statement is true. Our proof by contraposition succeeded; we have proved 
the theorem “If 3n + 2 is odd, then n is odd.” < 


Prove that if n = ab, where a and b are positive integers, then a < s/n orb< s/n. 


Solution: Because there is no obvious way of showing that a < s/n orb< s/n directly from the 
equation n = ab, where a and b are positive integers, we attempt a proof by contraposition. 
The first step in a proof by contraposition is to assume that the conclusion of the conditional 
statement “If n = ab, where a and b are positive integers, then a < s/n orb< yn” is false. That 
is, we assume that the statement (a < yn) V(b< s/n) is false. Using the meaning of disjunction 
together with De Morgan’s law, we see that this implies that both a < y/n and b < s/n are false. 
This implies that a > s/n and b > s/n. We can multiply these inequalities together (using the 


fact that if 0 < s < tand O < u < v, then su < tv) to obtain ab > yn . s/n = n. This shows that 
ab # n, which contradicts the statement n = ab. 

Because the negation of the conclusion of the conditional statement implies that the hy- 
pothesis is false, the original conditional statement is true. Our proof by contraposition suc- 


ceeded; we have proved that if n = ab, where a and b are positive integers, then a < s/n or 


b< vn. < 


VACUOUS AND TRIVIAL PROOFS We can quickly prove that a conditional statement 
p > q is true when we know that p is false, because p > q must be true when p is false. Con- 
sequently, if we can show that p is false, then we have a proof, called a vacuous proof, of the 
conditional statement p —> q. Vacuous proofs are often used to establish special cases of theo- 
rems that state that a conditional statement is true for all positive integers [i.e., a theorem of the 
kind VnP(n), where P(n) is a propositional function]. Proof techniques for theorems of this kind 
will be discussed in Section 5.1. 


Show that the proposition P(0) is true, where P(n) is “If n > 1, then n? > n” and the domain 
consists of all integers. 


Solution: Note that P(O) is “If 0 > 1, then 0? > 0.” We can show P(0) using a vacuous 
proof. Indeed, the hypothesis 0 > 1 is false. This tells us that P(O) is automatically true. < 


Remark: The fact that the conclusion of this conditional statement, 0? > 0, is false is irrelevant 
to the truth value of the conditional statement, because a conditional statement with a false 
hypothesis is guaranteed to be true. 


Prove that if n is an integer with 10 < n < 15 which is a perfect square, then n is also a perfect 
cube. 


Solution: Note that there are no perfect squares n with 10 < n < 15, because 32 = 9 and 42 = 16. 
Hence, the statement that n is an integer with 10 < n < 15 which is a perfect square is false for 
all integers n. Consequently, the statement to be proved is true for all integers n. < 
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We can also quickly prove a conditional statement p > q if we know that the conclusion 
q is true. By showing that q is true, it follows that p > q must also be true. A proof of p > q 
that uses the fact that q is true is called a trivial proof. Trivial proofs are often important when 
special cases of theorems are proved (see the discussion of proof by cases in Section 1.8) and 
in mathematical induction, which is a proof technique discussed in Section 5.1. 


Let P(n) be “If a and b are positive integers with a > b, then a” > b”, where the domain consists 
of all nonnegative integers. Show that P(O) is true. 


Solution: The proposition P(0) is “If a > b, then a? > b°.” Because a? = b? = 1, the conclusion 
of the conditional statement “If a > b, then a? > b°” is true. Hence, this conditional statement, 
which is P(O), is true. This is an example of a trivial proof. Note that the hypothesis, which is 
the statement “a > b,” was not needed in this proof. < 


A LITTLE PROOF STRATEGY We have described two important approaches for proving 
theorems of the form Vx(P(x) > Q(x)): direct proof and proof by contraposition. We have also 
given examples that show how each is used. However, when you are presented with a theorem 
of the form Vx(P(x) > Q(x)), which method should you use to attempt to prove it? We will 
provide a few rules of thumb here; in Section 1.8 we will discuss proof strategy at greater length. 

When you want to prove a statement of the form Vx(P(x) > Q(x)), first evaluate whether 
a direct proof looks promising. Begin by expanding the definitions in the hypotheses. Start to 
reason using these hypotheses, together with axioms and available theorems. If a direct proof 
does not seem to go anywhere, for instance when there is no clear way to use hypotheses as in 
Examples 3 and 4 to reach the conclusion, try the same thing with a proof by contraposition. 

(Hypotheses such as x is irrational or x £ 0 that are difficult to reason from are a clue that 
an indirect proof might be your best best.) 

Recall that in a proof by contraposition you assume that the conclusion of the conditional 
statement is false and use a direct proof to show this implies that the hypothesis must be false. 
Often, you will find that a proof by contraposition is easily constructed from the negation of 
the conclusion. We illustrate this strategy in Examples 7 and 8. In each example, note how 
straightforward a proof by contraposition is, while there is no clear way to provide a direct 
proof. 

Before we present our next example, we need a definition. 


The real number r is rational if there exist integers p and q with q 4 0 such that r = p/q. A 
real number that is not rational is called irrational. 


Prove that the sum of two rational numbers is rational. (Note that if we include the implicit 
quantifiers here, the theorem we want to prove is “For every real number r and every real number 
s, if r and s are rational numbers, then r + s is rational.) 


Solution: We first attempt a direct proof. To begin, suppose that r and s are rational numbers. 
From the definition of a rational number, it follows that there are integers p and q, with q + 0, 
such that r = p/q, and integers t and u, with u # 0, such that s = t/u. Can we use this informa- 
tion to show that r + s is rational? That is, can we find integers v and w such that r + s = v/w 
and w Æ 0? 

With the goal of finding these integers v and w, we add r = p/q and s = t/u, using qu as 
the common denominator. We find that 


P 


ut+qt 
jee ees 
q 


t 
u qu 
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Because q £ 0 and u ¥ 0, it follows that gu + 0. Consequently, we have expressed r + s as the 
ratio of two integers, v = pu + qt and w = qu, where w £ 0. This means that r + s is rational. 
We have proved that the sum of two rational numbers is rational; our attempt to find a direct 
proof succeeded. < 


Prove that if n is an integer and n? is odd, then n is odd. 


Solution: We first attempt a direct proof. Suppose that n is an integer and n? is odd. From the 
definition of an odd integer, there exists an integer k such that n? = 2k + 1. Can we use this 
information to show that n is odd? There seems to be no obvious approach to show that n is odd 


because solving for n produces the equation n = + 2k + 1, which is not terribly useful. 
Because this attempt to use a direct proof did not bear fruit, we next attempt a proof by 
contraposition. We take as our hypothesis the statement that n is not odd. Because every integer 
is odd or even, this means that n is even. This implies that there exists an integer k such that 
n = 2k. To prove the theorem, we need to show that this hypothesis implies the conclusion 
that n? is not odd, that is, that n? is even. Can we use the equation n = 2k to achieve this? By 
squaring both sides of this equation, we obtain n? = 4k? = 2(2k*), which implies that n? is also 
even because n? = 2t, where t = 2k*. We have proved that if n is an integer and n? is odd, then 
n is odd. Our attempt to find a proof by contraposition succeeded. < 


1.7.7 Proofs by Contradiction 


Suppose we want to prove that a statement p is true. Furthermore, suppose that we can find a 
contradiction q such that =p —> q is true. Because q is false, but =p — q is true, we can conclude 
that =p is false, which means that p is true. How can we find a contradiction q that might help 
us prove that p is true in this way? 

Because the statement r A ^r is a contradiction whenever r is a proposition, we can prove 
that p is true if we can show that =p > (r A ^r) is true for some proposition r. Proofs of this type 
are called proofs by contradiction. Because a proof by contradiction does not prove a result 
directly, itis another type of indirect proof. We provide three examples of proof by contradiction. 
The first is an example of an application of the pigeonhole principle, a combinatorial technique 
that we will cover in depth in Section 6.2. 


Show that at least four of any 22 days must fall on the same day of the week. 


Solution: Let p be the proposition “At least four of 22 chosen days fall on the same day of the 
week.” Suppose that ~p is true. This means that at most three of the 22 days fall on the same 
day of the week. Because there are seven days of the week, this implies that at most 21 days 
could have been chosen, as for each of the days of the week, at most three of the chosen days 
could fall on that day. This contradicts the premise that we have 22 days under consideration. 
That is, if r is the statement that 22 days are chosen, then we have shown that =p > (r A ^r). 
Consequently, we know that p is true. We have proved that at least four of 22 chosen days fall 
on the same day of the week. < 


Prove that y2 is irrational by giving a proof by contradiction. 


Solution: Let p be the proposition «4/2 is irrational.” To start a proof by contradiction, we sup- 
pose that ~p is true. Note that =p is the statement “It is not the case that V2 is irrational,” which 
says that V2 is rational. We will show that assuming that ~p is true leads to a contradiction. 


EXAMPLE 12 


1.7 Introduction to Proofs 91 


If V2 is rational, there exist integers a and b with V2 = a/b, where b + 0 and a and b have 
no common factors (so that the fraction a/b is in lowest terms). (Here, we are using the fact that 


every rational number can be written in lowest terms.) Because v2 = a/b, when both sides of 
this equation are squared, it follows that 


a 
Hence, 
2b? = a 


By the definition of an even integer it follows that a? is even. We next use the fact that if a? is 
even, a must also be even, which follows by Exercise 18. Furthermore, because a is even, by 
the definition of an even integer, a = 2c for some integer c. Thus, 


Dividing both sides of this equation by 2 gives 
b= Der. 


By the definition of even, this means that b? is even. Again using the fact that if the square of 
an integer is even, then the integer itself must be even, we conclude that b must be even as well. 


We have now shown that the assumption of ~p leads to the equation V2 = a/b, where 
a and b have no common factors, but both a and b are even, that is, 2 divides both a and b. 
Note that the statement that /2 = a/b, where a and b have no common factors, means, in 
particular, that 2 does not divide both a and b. Because our assumption of =p leads to the 
contradiction that 2 divides both a and b and 2 does not divide both a and b, =p must be false. 


That is, the statement p, “4/2 is irrational,” is true. We have proved that V2 is irrational. < 


Proof by contradiction can be used to prove conditional statements. In such proofs, we first 
assume the negation of the conclusion. We then use the premises of the theorem and the negation 
of the conclusion to arrive at a contradiction. (The reason that such proofs are valid rests on the 
logical equivalence of p > q and (p A nq) > F. To see that these statements are equivalent, 
simply note that each is false in exactly one case, namely, when p is true and q is false.) 

Note that we can rewrite a proof by contraposition of a conditional statement as a proof 
by contradiction. In a proof of p > q by contraposition, we assume that ~g is true. We then 
show that =p must also be true. To rewrite a proof by contraposition of p > q as a proof by 
contradiction, we suppose that both p and ~q are true. Then, we use the steps from the proof of 
~q — =p to show that =p is true. This leads to the contradiction p A =p, completing the proof. 
Example 11 illustrates how a proof by contraposition of a conditional statement can be rewritten 
as a proof by contradiction. 


Give a proof by contradiction of the theorem “If 37n + 2 is odd, then n is odd.” 


Solution: Let p be “3n +2 is odd” and q be “n is odd.” To construct a proof by contradiction, 
assume that both p and ~q are true. That is, assume that 37 + 2 is odd and that n is not odd. 
Because n is not odd, we know that it is even. Because n is even, there is an integer k such 
that n = 2k. This implies that 3n + 2 = 3(2k) + 2 = 6k + 2 = 2(3k + 1). Because 3n + 2 is 2t, 
where t = 3k + 1, 3n + 2 is even. Note that the statement “3n + 2 is even” is equivalent to the 
statement =p, because an integer is even if and only if it is not odd. Because both p and =p are 
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true, we have a contradiction. This completes the proof by contradiction, proving that if 3n + 2 
is odd, then 7 is odd. < 


Note that we can also prove by contradiction that p > q is true by assuming that p and ~q 
are true, and showing that q must be also be true. This implies that sg and q are both true, a 
contradiction. This observation tells us that we can turn a direct proof into a proof by contra- 
diction. 


PROOFS OF EQUIVALENCE To prove a theorem that is a biconditional statement, that is, a 
statement of the form p © q, we show that p > q and q > p are both true. The validity of this 
approach is based on the tautology 


peq) (p >q) ^q >p). 


Prove the theorem “If n is an integer, then n is odd if and only if n? is odd.” 


Solution: This theorem has the form “p if and only if q,” where p is “n is odd” and q is “n? 
is odd.” (As usual, we do not explicitly deal with the universal quantification.) To prove this 
theorem, we need to show that p > q and q > p are true. 

We have already shown (in Example 1) that p > q is true and (in Example 8) that q > p is 
true. 

Because we have shown that both p > q and q > p are true, we have shown that the theorem 
is true. < 


Sometimes a theorem states that several propositions are equivalent. Such a theorem states 
that propositions p4, P2, P3» -.., P, are equivalent. This can be written as 


Pi S Pa S = PF Py 


which states that all n propositions have the same truth values, and consequently, that for all i 
and j with 1 <i < nand1 <j < n, p; and p; are equivalent. One way to prove these are mutually 
equivalent is to use the tautology 


Pi © P2 oo e Pn -> (Pı > Pr) A (Pz > p3) A A (Pn > pı). 


This shows that if the n conditional statements p) > p>, P2 > P3, ---, Pn > pı can be shown to 
be true, then the propositions pj, p>, ..., p, are all equivalent. 


This is much more efficient than proving that p; > Pj for all i Aj with 1 < i < n and 1 < 
j <n. (Note that there are n? — n such conditional statements.) 

When we prove that a group of statements are equivalent, we can establish any chain of 
conditional statements we choose as long as it is possible to work through the chain to go from 
any one of these statements to any other statement. For example, we can show that p,, p), and 


p, are equivalent by showing that p, > p3, P > P2, and py > p}. 
Show that these statements about the integer n are equivalent: 


Pi: nis even. 
Py: n-— lis odd. 


p3: n? is even. 


Solution: We will show that these three statements are equivalent by showing that the condi- 
tional statements p} > p2, P2 > p3, and p} —> p, are true. 
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We use a direct proof to show that p} > p,. Suppose that n is even. Then n = 2k for some 
integer k. Consequently, n — 1 = 2k — 1 = 2(k — 1) + 1. This means that n — 1 is odd because 
it is of the form 2m + 1, where m is the integer k — 1. 

We also use a direct proof to show that p, > p,. Now suppose n-— 1 is odd. Then 
n — 1 = 2k + 1 for some integer k. Hence, n = 2k + 2 so that n? = (2k + 2)? = 4k? + 8k + 4 = 
2(2k* + 4k + 2). This means that n? is twice the integer 2k? + 4k + 2, and hence is even. 

To prove p, > pı, we use a proof by contraposition. That is, we prove that if n is not even, 
then n? is not even. This is the same as proving that if n is odd, then n? is odd, which we have 
already done in Example 1. This completes the proof. < 


COUNTEREXAMPLES [In Section 1.4 we stated that to show that a statement of the form 
VxP(x) is false, we need only find a counterexample, that is, an example x for which P(x) is 
false. When presented with a statement of the form VxP(x), which we believe to be false or 
which has resisted all proof attempts, we look for a counterexample. We illustrate the use of 
counterexamples in Example 15. 


Show that the statement “Every positive integer is the sum of the squares of two integers” is false. 


Solution: To show that this statement is false, we look for a counterexample, which is a par- 
ticular integer that is not the sum of the squares of two integers. It does not take long to find 
a counterexample, because 3 cannot be written as the sum of the squares of two integers. To 
show this is the case, note that the only perfect squares not exceeding 3 are 0? = 0 and 1? = 1. 
Furthermore, there is no way to get 3 as the sum of two terms each of which is 0 or 1. Conse- 
quently, we have shown that “Every positive integer is the sum of the squares of two integers” 
is false. < 


1.7.8 Mistakes in Proofs 


There are many common errors made in constructing mathematical proofs. We will briefly de- 
scribe some of these here. Among the most common errors are mistakes in arithmetic and basic 
algebra. Even professional mathematicians make such errors, especially when working with 
complicated formulae. Whenever you use such computations you should check them as care- 
fully as possible. (You should also review any troublesome aspects of basic algebra, especially 
before you study Section 5.1.) 

Each step of a mathematical proof needs to be correct and the conclusion needs to follow 
logically from the steps that precede it. Many mistakes result from the introduction of steps that 
do not logically follow from those that precede it. This is illustrated in Examples 16-18. 


What is wrong with this famous supposed “proof” that 1 = 2? 


“Proof”: We use these steps, where a and b are two equal positive integers. 


Step Reason 

l.a=b Given 

2. @ =ab Multiply both sides of (1) by a 

3. @ — b? = ab — b? Subtract b? from both sides of (2) 

4. (a—b)(at+b) = b(a — b) Factor both sides of (3) 

5.a+b=b Divide both sides of (4) by a — b 

6. 2b=b Replace a by b in (5) because a = b 
and simplify 


7.2=1 Divide both sides of (6) by b 
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Solution: Every step is valid except for step 5, where we divided both sides by a — b. The error 
is that a — b equals zero; division of both sides of an equation by the same quantity is valid as 
long as this quantity is not zero. < 


What is wrong with this “proof”? 


“Theorem”: If n? is positive, then n is positive. 


“Proof”: Suppose that n? is positive. Because the conditional statement “If n is positive, then 
n is positive” is true, we can conclude that n is positive. 


Solution: Let P(n) be “n is positive” and Q(n) be “‘n* is positive.” Then our hypothesis is Q(7). 
The statement “If n is positive, then n? is positive” is the statement Vn(P(n) > Q(n)). From 
the hypothesis Q(n) and the statement Vn(P(n) > Q(n)) we cannot conclude P(n), because we 
are not using a valid rule of inference. Instead, this is an example of the fallacy of affirming 
the conclusion. A counterexample is supplied by n = —1 for which n? = 1 is positive, but n is 
negative. < 


What is wrong with this “proof”? 


“Theorem”: If n is not positive, then n? is not positive. (This is the contrapositive of the 
“theorem” in Example 17.) 


“Proof”: Suppose that n is not positive. Because the conditional statement “If n is positive, 
then n? is positive” is true, we can conclude that n? is not positive. 


Solution: Let P(n) and Q(n) be as in the solution of Example 17. Then our hypothesis is =P(7) 
and the statement “If n is positive, then n? is positive” is the statement Vn(P(n) > Q(n)). From 
the hypothesis =P() and the statement Vn(P(n) > Q(n)) we cannot conclude =Q(n), because 
we are not using a valid rule of inference. Instead, this is an example of the fallacy of denying 
the hypothesis. A counterexample is supplied by n = —1, as in Example 17. < 


Finally, we briefly discuss a particularly nasty type of error. Many incorrect arguments are 
based on a fallacy called begging the question. This fallacy occurs when one or more steps of 
a proof are based on the truth of the statement being proved. In other words, this fallacy arises 
when a statement is proved using itself, or a statement equivalent to it. That is why this fallacy 
is also called circular reasoning. 


Is the following argument correct? It supposedly shows that n is an even integer whenever n? is 


an even integer. 


Suppose that n? is even. Then n? = 2k for some integer k. Let n = 2/ for some integer l. This 
shows that n is even. 


Solution: This argument is incorrect. The statement “let n = 2/ for some integer /” occurs in the 
proof. No argument has been given to show that n can be written as 2/ for some integer l. This is 
circular reasoning because this statement is equivalent to the statement being proved, namely, 
“n is even.” The result itself is correct; only the method of proof is wrong. < 
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Making mistakes in proofs is part of the learning process. When you make a mistake that 
someone else finds, you should carefully analyze where you went wrong and make sure that 
you do not make the same mistake again. Even professional mathematicians make mistakes in 
proofs. More than a few incorrect proofs of important results have fooled people for many years 
before subtle errors in them were found. 


1.7.9 Just a Beginning 


We have now developed a basic arsenal of proof methods. In the next section we will introduce 
other important proof methods. We will also introduce several important proof techniques in 
Chapter 5, including mathematical induction, which can be used to prove results that hold for 
all positive integers. In Chapter 6 we will introduce the notion of combinatorial proofs. 

In this section we introduced several methods for proving theorems of the form Vx(P(x) > 
Q(x)), including direct proofs and proofs by contraposition. There are many theorems of this type 
whose proofs are easy to construct by directly working through the hypotheses and definitions 
of the terms of the theorem. However, it is often difficult to prove a theorem without resorting 
to a clever use of a proof by contraposition or a proof by contradiction, or some other proof 
technique. In Section 1.8 we will address proof strategy. We will describe various approaches 
that can be used to find proofs when straightforward approaches do not work. Constructing 
proofs is an art that can be learned only through experience, including writing proofs, having 


your proofs critiqued, and reading and analyzing other proofs. 


Exercises 


12. 


13. 


. Use a direct proof to show that the sum of two odd inte- 


gers is even. 


. Use a direct proof to show that the sum of two even inte- 


gers is even. 


. Show that the square of an even number is an even num- 


ber using a direct proof. 


. Show that the additive inverse, or negative, of an even 


number and an irrational number is irrational. 


Prove that if x is irrational, then 1/x is irrational. 


14. 
15. 


Prove that if x is rational and x # 0, then 1/x is rational. 


Prove that if x is an irrational number and x > 0, then yx 
is also irrational. 


. Prove that if x, y, and z are integers and x + y + z is odd, 


then at least one of x, y, and z is odd. 


. Use a proof by contraposition to show that if x + y > 2, 


where x and y are real numbers, then x > 1 ory > 1. 


S ) c= 3 ; : 
number is an even number using a direct proof. 18. Prove that ifm and n are integers and mn is even, then m 
a har if d int h is even or n is even. 

eT Lat 1 MEM and on aTe EVEN eee 19. Show that if n is an integer and n? + 5 is odd, then n is 
m, n, and p are integers, then m + p is even. What kind of $ 

roof did you use? AEE 
P : ; a) a proof by contraposition. 

. Use a direct proof to show that the product of two odd b) a proof by contradiction. 
numbers is odd. 20. Prove that if n is an integer and 3n + 2 is even, then n is 

. Use a direct proof to show that every odd integer is the even using 
difference of two squares. [Hint: Find the difference of a) a proof by contraposition. 
the squares of k + 1 and k where k is a positive integer. ] b) a proof by contradiction. 

. Prove that if n is a perfect square, then n + 2 is not a per- 21. Prove the proposition P(0), where P(n) is the proposition 
fect square. “If n is a positive integer greater than 1, then n? > n.” 

. Use a proof by contradiction to prove that the sum of an What kind of proof did you use? 
irrational number and a rational number is irrational. 22. Prove the proposition P(1), where 3 (n) is the proposi- 

. Use a direct proof to show that the product of two rational ton apa a P e integer, then n° > n.” What kind of 
numbers is rational. PEQOR CIG YOU USE B . 

: PP 23. Let P(n) be the proposition “If a and b are positive real 

. Prove or disprove that the product of two irrational num- numbers, then (a+ b)" >a" +b".” Prove that P(1) is 
bers is tena f true. What kind of proof did you use? 

Prove or disprove that the product of a nonzero rational 24. Show that if you pick three socks from a drawer contain- 


ing just blue socks and black socks, you must get either a 
pair of blue socks or a pair of black socks. 
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25. 
26. 


27. 


28. 
29. 


30. 
31. 


32. 


33. 
34. 
35. 


36. 
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Show that at least ten of any 64 days chosen must fall on 
the same day of the week. 


Show that at least three of any 25 days chosen must fall 
in the same month of the year. 


Use a proof by contradiction to show that there is no ra- 
tional number r for which 7? + r+ 1 = 0. [Hint: Assume 
that r = a/b is aroot, where a and b are integers and a/b 
is in lowest terms. Obtain an equation involving integers 
by multiplying by 5°. Then look at whether a and b are 
each odd or even. |] 


Prove that if n is a positive integer, then n is even if and 
only if 7n + 4 is even. 
Prove that if n is a positive integer, then n is odd if and 
only if 5n + 6 is odd. 


Prove that m? = n? if and only if m = n or m = ~n. 
Prove or disprove that if m and n are integers such that 
mn = 1, then either m = 1 and n = 1, or else m = —1 and 
n=-l. 


Show that these three statements are equivalent, where a 
and b are real numbers: (i) a is less than b, (ii) the average 
of a and b is greater than a, and (iii) the average of a and 
bis less than b. 


Show that these statements about the integer x are 
equivalent: (7) 3x +2 is even, (ii) x +5 is odd, (iii) x? 
is even. 

Show that these statements about the real number x are 
equivalent: (i) x is rational, (ii) x/2 is rational, (iii) 3x — 1 
is rational. 

Show that these statements about the real number x are 
equivalent: (i) x is irrational, (ii) 3x +2 is irrational, 
(iii) x/2 is irrational. 

Is this reasoning for finding the solutions of the equa- 
tion V2x2 — 1 = x correct? (1) V2x2 — 1 = x is given; 


(2) 2x? — 1 = x’, obtained by squaring both sides of (1); 


37. 


38. 


39. 


40. 


41. 


42. 


43. 


44. 


(3) x? — 1 =0, obtained by subtracting x? from both 
sides of (2); (4) (x — 1)(x+ 1) = 0, obtained by factor- 
ing the left-hand side of x-1; (5) x=lorx=-1, 
which follows because ab = 0 implies that a=0 or 
b=0. 

Are these steps for finding the solutions of yx +3 = 
3 — x correct? (1) yx +3 = 3 — x is given; (2)x+3= 
x? — 6x + 9, obtained by squaring both sides of (1); (3) 
0 =x? — 7x + 6, obtained by subtracting x + 3 from both 
sides of (2); (4) 0 = (x — 1) — 6), obtained by factoring 
the right-hand side of (3); (5) x = 1 or x = 6, which fol- 
lows from (4) because ab = 0 implies thata = O or b = 0. 
Show that the propositions p,, p2, p3, and p4 can be shown 
to be equivalent by showing that p; © p4, P2 © p3, and 
Py  P3- 

Show that the propositions p}, P2 P3, P4» and ps; can 
be shown to be equivalent by proving that the condi- 


tional statements p; > Py, P3 > Pi» P4 > Pr» P2 > Ps» 
and p; > p} are true. 


Find a counterexample to the statement that every posi- 
tive integer can be written as the sum of the squares of 
three integers. 

Prove that at least one of the real numbers a}, dy, ..., a, 
is greater than or equal to the average of these numbers. 
What kind of proof did you use? 

Use Exercise 41 to show that if the first 10 positive inte- 
gers are placed around a circle, in any order, there exist 
three integers in consecutive locations around the circle 
that have a sum greater than or equal to 17. 

Prove that if is an integer, these four statements are 
equivalent: (i) n is even, (ii) n+ 1 is odd, (iii) 3n + 1 is 
odd, (iv) 3n is even. 

Prove that these four statements about the integer n are 
equivalent: (i) n? is odd, (ii) 1 — n is even, (iii) n? is odd, 
(iv) n? + 1 is even. 


1.8.1 Introduction 


Assessment > 


In Section 1.7 we introduced many methods of proof and illustrated how each method can 
be used. In this section we continue this effort. We will introduce several other commonly 


used proof methods, including the method of proving a theorem by considering different cases 
separately. We will also discuss proofs where we prove the existence of objects with desired 


properties. 


In Section 1.7 we briefly discussed the strategy behind constructing proofs. This strategy 
includes selecting a proof method and then successfully constructing an argument step by step, 
based on this method. In this section, after we have developed a versatile arsenal of proof meth- 
ods, we will study some aspects of the art and science of proofs. We will provide advice on how 
to find a proof of a theorem. We will describe some tricks of the trade, including how proofs 
can be found by working backward and by adapting existing proofs. 
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When mathematicians work, they formulate conjectures and attempt to prove or disprove 
them. We will briefly describe this process here by proving results about tiling checkerboards 
with dominoes and other types of pieces. Looking at tilings of this kind, we will be able to 
quickly formulate conjectures and prove theorems without first developing a theory. 

We will conclude the section by discussing the role of open questions. In particular, we 
will discuss some interesting problems either that have been solved after remaining open for 
hundreds of years or that still remain open. 


1.8.2 Exhaustive Proof and Proof by Cases 


Sometimes we cannot prove a theorem using a single argument that holds for all possible cases. 
We now introduce a method that can be used to prove a theorem by considering different cases 
separately. This method is based on a rule of inference that we will now introduce. To prove a 
conditional statement of the form 


(Pi VP y s V Pn) >q 


the tautology 


[P1 VP Vt VYP) > q] e [pP > DAC >g) AA Aa > DI 


can be used as a rule of inference. This shows that the original conditional statement with a 
hypothesis made up of a disjunction of the propositions p,, p>, ..., P, can be proved by proving 
each of the n conditional statements p; > q, i = 1, 2, ...,n, individually. Such an argument is 
called a proof by cases. Sometimes to prove that a conditional statement p > q is true, it is 
convenient to use a disjunction p; V py V + V p, instead of p as the hypothesis of the conditional 
statement, where p and p; V p> V + V p, are equivalent. 


EXHAUSTIVE PROOF Some theorems can be proved by examining a relatively small num- 
ber of examples. Such proofs are called exhaustive proofs, or proofs by exhaustion because 
these proofs proceed by exhausting all possibilities. An exhaustive proof is a special type of 
proof by cases where each case involves checking a single example. We now provide some 
illustrations of exhaustive proofs. 


Prove that (n + 1} > 3" ifnisa positive integer with n < 4. 


Solution: We use a proof by exhaustion. We only need verify the inequality (n + 1)° > 3” when 
n = 1,2,3, and 4. For n = 1, we have (n + 1)? = 23 = 8 and 3” = 3! = 3; for n = 2, we have 
(n+ 1)? = 33 = 27 and 3” = 3? = 9; for n = 3, we have (n + 1)? = 4 = 64 and 3” = 33 = 27; 
and for n = 4, we have (n + 1)? = 5? = 125 and 3” = 34 = 81. In each of these four cases, we 
see that (n + 1) > 3”. We have used the method of exhaustion to prove that (n + 1} > 3" ifn 
is a positive integer with n < 4. < 


Prove that the only consecutive positive integers not exceeding 100 that are perfect powers are 8 
and 9. (An integer n is a perfect power if it equals m“, where m is an integer and a is an integer 
greater than 1.) 


Solution: We use a proof by exhaustion. In particular, we can prove this fact by examining 
positive integers n not exceeding 100, first checking whether n is a perfect power, and if it is, 
checking whether n + 1 is also a perfect power. A quicker way to do this is simply to look at all 
perfect powers not exceeding 100 and checking whether the next largest integer is also a perfect 
power. The squares of positive integers not exceeding 100 are 1, 4, 9, 16, 25, 36, 49, 64, 81, and 
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Proofs by exhaustion 
can tire out people and 
computers when the 
number of cases 
challenges the available 
processing power! 


EXAMPLE 3 
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EXAMPLE 4 


100. The cubes of positive integers not exceeding 100 are 1, 8, 27, and 64. The fourth powers 
of positive integers not exceeding 100 are 1, 16, and 81. The fifth powers of positive integers 
not exceeding 100 are 1 and 32. The sixth powers of positive integers not exceeding 100 are 1 
and 64. There are no powers of positive integers higher than the sixth power not exceeding 100, 
other than 1. Looking at this list of perfect powers not exceeding 100, we see that n = 8 is the 
only perfect power n for which n + 1 is also a perfect power. That is, 2° = 8 and 3? = 9 are the 
only two consecutive perfect powers not exceeding 100. < 


People can carry out exhaustive proofs when it is necessary to check only a relatively small 
number of instances of a statement. Computers do not complain when they are asked to check 
a much larger number of instances of a statement, but they still have limitations. Note that not 
even a computer can check all instances when it is impossible to list all instances to check. 


PROOF BY CASES A proof by cases must cover all possible cases that arise in a theorem. 
We illustrate proof by cases with a couple of examples. In each example, you should check that 
all possible cases are covered. 


Prove that if n is an integer, then n >n. 

Solution: We can prove that n? > n for every integer by considering three cases, when n = 0, 
when n > 1, and when n < —1. We split the proof into three cases because it is straightforward 
to prove the result by considering zero, positive integers, and negative integers separately. 


Case (i): When n = 0, because 0° = 0, we see that 0? > 0. It follows that n? > n is true in 
this case. 

Case (ii): When n > 1, when we multiply both sides of the inequality n > 1 by the positive 
integer n, we obtain n-n >n- 1. This implies that n2>nforn> 1. 

Case (iii): In this case n < —1. However, n? > 0. It follows that n? > n. 


Because the inequality n? > n holds in all three cases, we can conclude that if n is an integer, 
then n? > n. < 


Use a proof by cases to show that |xy| = |x||y|, where x and y are real numbers. (Recall that |a], 
the absolute value of a, equals a when a > 0 and equals —a when a < 0.) 


Solution: In our proof of this theorem, we remove absolute values using the fact that |a| = a 
when a > 0 and |a| = —a when a < 0. Because both |x| and |y| occur in our formula, we will 
need four cases: (i) x and y both nonnegative, (ii) x nonnegative and y negative, (iii) x negative 
and y nonnegative, and (iv) x negative and y negative. We denote by p,, po, p3, and p4, the 
proposition stating the assumption for each of these four cases, respectively. 

(Note that we can remove the absolute value signs by making the appropriate choice of 
signs within each case.) 


Case (i): We see that p} —> q because xy > 0 when x > 0 and y > 0, so that |xy| = xy = 
Ix|lyI. 

Case (ii): To see that p, > q, note that if x > O and y < 0, then xy < 0, so that |xy| = —xy = 
x(—y) = |x||y|. (Here, because y < 0, we have |y| = —y.) 

Case (iii): To see that p > q, we follow the same reasoning as the previous case with the 
roles of x and y reversed. 

Case (iv): To see that p, > q, note that when x < 0 and y < 0, it follows that xy > 0. Hence, 
Ixy] = xy = (—x)(-y) = |xllyl. 
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Because |xy| = |x||y| holds in each of the four cases and these cases exhaust all possibilities, 
we can conclude that |xy| = |x||y|, whenever x and y are real numbers. < 


LEVERAGING PROOF BY CASES The examples we have presented illustrating proof by 
cases provide some insight into when to use this method of proof. In particular, when it is not 
possible to consider all cases of a proof at the same time, a proof by cases should be considered. 
When should you use such a proof? Generally, look for a proof by cases when there is no obvious 
way to begin a proof, but when extra information in each case helps move the proof forward. 
Example 5 illustrates how the method of proof by cases can be used effectively. 


Formulate a conjecture about the final decimal digit of the square of an integer and prove your 
result. 


Solution: The smallest perfect squares are 1, 4,9, 16, 25, 36,49, 64, 81, 100, 121, 144, 169, 
196, 225, and so on. We notice that the digits that occur as the final digit of a square are 
0, 1, 4,5, 6, and 9, with 2, 3,7, and 8 never appearing as the final digit of a square. We con- 
jecture this theorem: The final decimal digit of a perfect square is 0, 1, 4, 5, 6, or 9. How can we 
prove this theorem? 

We first note that we can express an integer n as 10a + b, where a and b are positive inte- 
gers and b is 0, 1, 2, 3, 4, 5, 6, 7, 8, or 9. Here a is the integer obtained by subtracting the final 
decimal digit of n from n and dividing by 10. Next, note that (10a + b)? = 100a? + 20ab + b? = 
10(10a? + 2b) + b?, so that the final decimal digit of n? is the same as the final decimal digit 
of b?. Furthermore, note that the final decimal digit of b? is the same as the final decimal digit 
of (10 — b)? = 100 — 20b + b?. Consequently, we can reduce our proof to the consideration of 
Six cases. 


Case (i): The final digit of n is 1 or 9. Then the final decimal digit of n is the final decimal 
digit of 1? = 1 or 9? = 81, namely, 1. 

Case (ii): The final digit of n is 2 or 8. Then the final decimal digit of n? is the final decimal 
digit of 2? = 4 or 8? = 64, namely, 4. 

Case (iii): The final digit of n is 3 or 7. Then the final decimal digit of n? is the final decimal 
digit of 3? = 9 or 7? = 49, namely, 9. 

Case (iv): The final digit of n is 4 or 6. Then the final decimal digit of n? is the final decimal 
digit of 4? = 16 or 6? = 36, namely, 6. 

Case (v): The final decimal digit of n is 5. Then the final decimal digit of n? is the final 
decimal digit of 5? = 25, namely, 5. 

Case (vi): The final decimal digit of n is 0. Then the final decimal digit of n? is the final 
decimal digit of 0? = 0, namely, 0. 


Because we have considered all six cases, we can conclude that the final decimal digit of n?, 
where n is an integer is either 0, 1, 2, 4, 5, 6, or 9. < 


Sometimes we can eliminate all but a few examples in a proof by cases, as Example 6 
illustrates. 


Show that there are no solutions in integers x and y of x? + 3y? = 8. 


Solution: We can quickly reduce a proof to checking just a few simple cases because x? > 8 
when |x| > 3 and 3y? > 8 when |y| > 2. This leaves the cases when x equals —2, —1, 0, 1, or 2 
and y equals —1, 0, or 1. We can finish using an exhaustive proof. To dispense with the remaining 
cases, we note that possible values for x? are 0, 1, and 4, and possible values for 3y? are 0 and 
3, and the largest sum of possible values for x? and 3y? is 7. Consequently, it is impossible for 
xX? + 3y? = 8 to hold when x and y are integers. < 
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In a proof by cases be 
sure not to omit any 
cases and check that you 
have proved all cases 
correctly! 


EXAMPLE 7 


EXAMPLE 8 


WITHOUT LOSS OF GENERALITY In the proof in Example 4, we dismissed case (iii), 
where x < 0 and y > 0, because it is the same as case (ii), where x > 0 and y < 0, with the roles 
of x and y reversed. To shorten the proof, we could have proved cases (ii) and (iii) together by 
assuming, without loss of generality, that x > 0 and y < 0. Implicit in this statement is that we 
can complete the case with x < 0 and y > 0 using the same argument as we used for the case 
with x > 0 and y < 0, but with the obvious changes. 

In general, when the phrase “without loss of generality” is used in a proof (often abbre- 
viated as WLOG), we assert that by proving one case of a theorem, no additional argument is 
required to prove other specified cases. That is, other cases follow by making straightforward 
changes to the argument, or by filling in some straightforward initial step. Proofs by cases can 
often be made much more efficient when the notion of without loss of generality is employed. 
Incorrect use of this principle, however, can lead to unfortunate errors. Sometimes assumptions 
are made that lead to a loss in generality. Such assumptions can be made that do not take into 
account that one case may be substantially different from others. This can lead to an incomplete, 
and possibly unsalvageable, proof. In fact, many incorrect proofs of famous theorems turned 
out to rely on arguments that used the idea of “without loss of generality” to establish cases 
that could not be quickly proved from simpler cases. 

We now illustrate a proof where without loss of generality is used effectively together with 
other proof techniques. 


Show that if x and y are integers and both xy and x + y are even, then both x and y are even. 


Solution: We will use proof by contraposition, the notion of without loss of generality, and proof 
by cases. First, suppose that x and y are not both even. That is, assume that x is odd or that y is 
odd (or both). Without loss of generality, we assume that x is odd, so that x = 2m + 1 for some 
integer k. 

To complete the proof, we need to show that xy is odd or x+ y is odd. Consider two 
cases: (i) y is even, and (ii) y is odd. In (i), y= 2n for some integer n, so that x+y = 
(2m + 1) + 2n = 2(m +n) + 1 is odd. In (ii), y = 2n + 1 for some integer n, so that xy = (2m + 
1)(2n + 1) = 4mn + 2m + 2n + 1 = 202mn+m-+n)+1 is odd. This completes the proof by 
contraposition. (Note that our use of without loss of generality within the proof is justified 
because the proof when y is odd can be obtained by simply interchanging the roles of x and y in 
the proof we have given.) < 


COMMON ERRORS WITH EXHAUSTIVE PROOF AND PROOF BY CASES A common 
error of reasoning is to draw incorrect conclusions from examples. No matter how many separate 
examples are considered, a theorem is not proved by considering examples unless every possible 
case is covered. The problem of proving a theorem is analogous to showing that a computer 
program always produces the output desired. No matter how many input values are tested, unless 
all input values are tested, we cannot conclude that the program always produces the correct 
output. 


Is it true that every positive integer is the sum of 18 fourth powers of integers? 


Solution: To determine whether a positive integer n can be written as the sum of 18 fourth 
powers of integers, we might begin by examining whether n is the sum of 18 fourth powers of 
integers for the smallest positive integers. Because the fourth powers of integers are 0, 1, 16, 
81, ..., if we can select 18 terms from these numbers that add up to n, then n is the sum of 
18 fourth powers. We can show that all positive integers up to 78 can be written as the sum of 
18 fourth powers. (The details are left to the reader.) However, if we decided this was enough 
checking, we would come to the wrong conclusion. It is not true that every positive integer is 


EXAMPLE 9 


EXAMPLE 10 


Extra » 
Examples 


EXAMPLE 11 


1.8 Proof Methods and Strategy 101 


the sum of 18 fourth powers because 79 is not the sum of 18 fourth powers (as the reader can 
verify). < 


Another common error involves making unwarranted assumptions that lead to incorrect 
proofs by cases where not all cases are considered. This is illustrated in Example 9. 


What is wrong with this “proof”? 


“Theorem”: If x is a real number, then x” is a positive real number. 


“Proof”: Let p; be “x is positive,” let p, be “x is negative,” and let q be “x* is positive.” To 
show that p, > q is true, note that when x is positive, x” is positive because it is the product of 
two positive numbers, x and x. To show that p, > q, note that when x is negative, x? is positive 
because it is the product of two negative numbers, x and x. This completes the proof. 


Solution: The problem with this “proof” is that we missed the case of x = 0. When x = 0, 
x? = 0 is not positive, so the supposed theorem is false. If p is “x is a real number,” then 
we can prove results where p is the hypothesis with three cases, pı, pọ, and p}, where 
pı is “x is positive,” p, is “x is negative,’ and p} is “x =0” because of the equivalence 
P © Pi V Po V P3- < 


‘ é 


1.8.3 Existence Proofs 


Many theorems are assertions that objects of a particular type exist. A theorem of this type is 
a proposition of the form 3xP(x), where P is a predicate. A proof of a proposition of the form 
AxP(x) is called an existence proof. There are several ways to prove a theorem of this type. 
Sometimes an existence proof of 4xP(x) can be given by finding an element a, called a witness, 
such that P(a) is true. This type of existence proof is called constructive. It is also possible 
to give an existence proof that is nonconstructive; that is, we do not find an element a such 
that P(a) is true, but rather prove that 4xP(x) is true in some other way. One common method 
of giving a nonconstructive existence proof is to use proof by contradiction and show that the 
negation of the existential quantification implies a contradiction. The concept of a constructive 
existence proof is illustrated by Example 10 and the concept of a nonconstructive existence 
proof is illustrated by Example 11. 


A Constructive Existence Proof Show that there is a positive integer that can be written as 
the sum of cubes of positive integers in two different ways. 


Solution: After considerable computation (such as a computer search) we find that 
1729 =10° +9 =127 +1. 


Because we have displayed a positive integer that can be written as the sum of cubes in two 
different ways, we are done. 

There is an interesting story pertaining to this example. The English mathematician G. H. 
Hardy, when visiting the ailing Indian prodigy Ramanujan in the hospital, remarked that 1729, 
the number of the cab he took, was rather dull. Ramanujan replied “No, it is a very interesting 
number; it is the smallest number expressible as the sum of cubes in two different ways.” < 


A Nonconstructive Existence Proof Show that there exist irrational numbers x and y such 
that x” is rational. 
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Solution: By Example 11 in Section 1.7 we know that V2 is irrational. Consider the number 
a” If it is rational, we have two irrational numbers x and y with x’ rational, namely, x = /2 
and y = /2. On the other hand if g” is irrational, then we can let x = ie and y = 2 so 
that x” = asi = pan = V2 =2. 


This proof is an example of a nonconstructive existence proof because we have not found 
irrational numbers x and y such that x” is rational. Rather, we have shown that either the pair 


v2 
x= V2, y= V2 or the pair x = V2 = V2 have the desired property, but we do not know 
which of these two pairs works! < 


Remark: Exercise 11 in Section 4.3 provides a constructive existence proof that there are irra- 
tional numbers x and y such that x” is rational. 


Nonconstructive existence proofs often are quite subtle, as Example 12 illustrates. 


EXAMPLE 12 Chompisa game played by two players. In this game, cookies are laid out on a rectangular grid. 
The cookie in the top left position is poisoned, as shown in Figure 1(a). The two players take 
turns making moves; at each move, a player is required to eat a remaining cookie, together with 

Links » all cookies to the right and/or below it (see Figure 1(b), for example). The loser is the player 
who has no choice but to eat the poisoned cookie. We ask whether one of the two players has a 
winning strategy. That is, can one of the players always make moves that are guaranteed to lead 
to a win? 


GODFREY HAROLD HARDY (1877-1947) Hardy, born in Cranleigh, Surrey, England, was the older of 
two children of Isaac Hardy and Sophia Hall Hardy. His father was the geography and drawing master at the 
Cranleigh School and also gave singing lessons and played soccer. His mother gave piano lessons and helped run 
a boardinghouse for young students. Hardy’s parents were devoted to their children’s education. Hardy demon- 
strated his numerical ability at the early age of two when he began writing down numbers into the millions. 
He had a private mathematics tutor rather than attending regular classes at the Cranleigh School. He moved to 
Winchester College, a private high school, when he was 13 and was awarded a scholarship. He excelled in his 
studies and demonstrated a strong interest in mathematics. He entered Trinity College, Cambridge, in 1896 on 
Error: a scholarship and won several prizes during his time there, graduating in 1899. 
PHILOSOPHICAL Hardy held the position of lecturer in mathematics at Trinity College at Cambridge University from 
SOCIETY/Science Source 1906 to 1919, when he was appointed to the Sullivan chair of geometry at Oxford. He had become un- 
happy with Cambridge over the dismissal of the noted philosopher and mathematician Bertrand Russell 
from Trinity for antiwar activities and did not like a heavy load of administrative duties. In 1931 he returned to Cambridge as the 
Sadleirian professor of pure mathematics, where he remained until his retirement in 1942. He was a pure mathematician and held 
an elitist view of mathematics, hoping that his research could never be applied. Ironically, he is perhaps best known as one of the 
developers of the Hardy—Weinberg law, which predicts patterns of inheritance. His work in this area appeared as a letter to the 
journal Science in which he used simple algebraic ideas to demonstrate errors in an article on genetics. Hardy worked primarily 
in number theory and function theory, exploring such topics as the Riemann zeta function, Fourier series, and the distribution of 
primes. He made many important contributions to many important problems, such as Waring’s problem about representing positive 
integers as sums of kth powers and the problem of representing odd integers as sums of three primes. Hardy is also remembered 
for his collaborations with John E. Littlewood, a colleague at Cambridge, with whom he wrote more than 100 papers, and the 
celebrated Indian mathematical prodigy Srinivasa Ramanujan. His collaboration with Littlewood led to the joke that there were only 
three important English mathematicians at that time, Hardy, Littlewood, and Hardy—Littlewood. However, some people believed 
that Hardy had invented a fictitious person, Littlewood, because Littlewood was seldom seen away from Cambridge. Hardy had 
the wisdom of recognizing Ramanujan’s genius from unconventional but extremely creative writings Ramanujan sent him, while 
other mathematicians failed to see the genius. Hardy brought Ramanujan to Cambridge and collaborated on important joint papers, 
establishing new results on the number of partitions of an integer. Hardy was interested in mathematics education, and his book 
A Course of Pure Mathematics had a profound effect on undergraduate instruction in mathematics in the first half of the twentieth 
century. Hardy also wrote A Mathematician’s Apology, in which he gives his answer to the question of whether it is worthwhile to 
devote one’s life to the study of mathematics. It presents Hardy’s view of what mathematics is and what a mathematician does. 
Hardy had a strong interest in sports. He was an avid cricket fan and followed scores closely. One peculiar trait he had was that 
he did not like his picture taken (only five snapshots are known) and disliked mirrors, covering them with towels immediately upon 
entering a hotel room. 
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FIGURE 1 (a) Chomp (top left cookie poisoned). (b) Three possible moves. 


EXAMPLE 13 


Solution: We will give a nonconstructive existence proof of a winning strategy for the first 
player. That is, we will show that the first player always has a winning strategy without explicitly 
describing the moves this player must follow. 

First, note that the game ends and cannot finish in a draw because with each move at least 
one cookie is eaten, so after no more than m X n moves the game ends, where the initial grid is 
m Xn. Now, suppose that the first player begins the game by eating just the cookie in the bottom 
right corner. There are two possibilities, this is the first move of a winning strategy for the first 
player, or the second player can make a move that is the first move of a winning strategy for the 
second player. In this second case, instead of eating just the cookie in the bottom right corner, 
the first player could have made the same move that the second player made as the first move of 
a winning strategy (and then continued to follow that winning strategy). This would guarantee 
a win for the first player. 

Note that we showed that a winning strategy exists, but we did not specify an actual winning 
strategy. Consequently, the proof is a nonconstructive existence proof. In fact, no one has been 
able to describe a winning strategy for Chomp that applies for all rectangular grids by describing 
the moves that the first player should follow. However, winning strategies can be described for 
certain special cases, such as when the grid is square and when the grid only has two rows of 
cookies (see Exercises 15 and 16 in Section 5.2). < 


1.8.4 Uniqueness Proofs 


Some theorems assert the existence of a unique element with a particular property. In other 
words, these theorems assert that there is exactly one element with this property. To prove a 
statement of this type we need to show that an element with this property exists and that no 
other element has this property. The two parts of a uniqueness proof are: 


Existence: We show that an element x with the desired property exists. 
Uniqueness: We show that if x and y both have the desired property, then x = y. 


Remark: Showing that there is a unique element x such that P(x) is the same as proving the 
statement 4x(P(x) A Yy Æ x > 7P(y))). 


We illustrate the elements of a uniqueness proof in Example 13. 


Show that if a and b are real numbers and a +Æ 0, then there is a unique real number r such that 
ar+b=0. 
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Solution: First, note that the real number r= —b/a is a solution of ar+b=0 because 
a(—b/a) + b = —b +b = 0. Consequently, a real number r exists for which ar + b = 0. This 

is the existence part of the proof. 
Second, suppose that s is a real number such that as + b = 0. Then ar + b = as + b, where 
Extra > r = —b/a. Subtracting b from both sides, we find that ar = as. Dividing both sides of this last 
Examples equation by a, which is nonzero, we see that r = s. This establishes the uniqueness part of the 
proof. < 


1.8.5 Proof Strategies 


Finding proofs can be a challenging business. When you are confronted with a statement to 
prove, you should first replace terms by their definitions and then carefully analyze what the 
hypotheses and the conclusion mean. After doing so, you can attempt to prove the result us- 
ing one of the available methods of proof. We have already provided some proof strategies in 
Section 1.7 for theorems of the form Yx(P(x) > Q(x)), when we introduced direct proof, proof 
by contraposition, and proof by contradiction. If the statement is a conditional statement, try a 
direct proof first as long as the hypotheses provide a good starting point; if this fails, try a proof 
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SRINIVASA RAMANUJAN (1887-1920) The renowned mathematical prodigy Ramanujan was born and 
raised in southern India near the city of Madras (now called Chennai). His father was a clerk in a cloth shop. 
His mother contributed to the family income by singing at a local temple. Ramanujan studied at the local 
English language school, displaying his talent and interest for mathematics. At the age of 13 he mastered 
a textbook used by college students. When he was 15, a university student lent him a copy of Synopsis of 
Pure Mathematics. Ramanujan decided to work out the over 6000 results in this book, stated without proof 
or explanation, writing on sheets later collected to form notebooks. He graduated from high school in 1904, 
winning a scholarship to the University of Madras. Enrolling in a fine arts curriculum, he neglected his sub- 
@Nick Higham/Alamy Stock jects other than mathematics and lost his scholarship. He failed to pass examinations at the university four 
Photo : times from 1904 to 1907, doing well only in mathematics. During this time he filled his notebooks with orig- 

inal writings, sometimes rediscovering already published work and at other times making new discoveries. 

Without a university degree, it was difficult for Ramanujan to find a decent job. To survive, he had to depend on the goodwill 
of his friends. He tutored students in mathematics, but his unconventional ways of thinking and failure to stick to the syllabus caused 
problems. He was married in 1909 in an arranged marriage to a young woman nine years his junior. Needing to support himself and 
his wife, he moved to Madras and sought a job. He showed his notebooks of mathematical writings to his potential employers, but 
the books bewildered them. However, a professor at the Presidency College recognized his genius and supported him, and in 1912 
he found work as an accounts clerk, earning a small salary. 

Ramanujan continued his mathematical work during this time and published his first paper in 1910 in an Indian journal. He 
realized that his work was beyond that of the Indian mathematicians of his day and decided to write to leading English mathemati- 
cians. The first mathematicians he wrote to turned down his request for help. But in January 1913 he wrote to G. H. Hardy, who was 
inclined to turn Ramanujan down, but the mathematical statements in the letter, although stated without proof, puzzled Hardy. He 
decided to examine them closely with the help of his colleague and collaborator J. E. Littlewood. They decided, after careful study, 
that Ramanujan was probably a genius, because his statements “could only be written down by a mathematician of the highest class; 
they must be true, because if they were not true, no one would have the imagination to invent them.” 

Hardy arranged a scholarship for Ramanujan, bringing him to England in 1914. Hardy personally tutored him in mathematical 
analysis, and they collaborated for five years, proving significant theorems about the number of partitions of integers. During this 
time, Ramanujan made important contributions to number theory and also worked on continued fractions, infinite series, and elliptic 
functions. Ramanujan had amazing insight involving certain types of functions and series, but his purported theorems on prime 
numbers were often wrong, illustrating his vague idea of what constitutes a correct proof. He was one of the youngest members ever 
appointed a Fellow of the Royal Society. Unfortunately, in 1917 Ramanujan became extremely ill. At the time, it was thought that he 
had trouble with the English climate and had contracted tuberculosis. It is now thought that he suffered from a vitamin deficiency, 
brought on by Ramanujan’s strict vegetarianism and shortages in wartime England. He returned to India in 1919, continuing to 
do mathematics even when confined to his bed. He was religious and thought his mathematical talent came from his family deity, 
Namagiri. He considered mathematics and religion to be linked. He said that “an equation for me has no meaning unless it expresses 
a thought of God.” His short life came to an end in April 1920, when he was 32 years old. Ramanujan left several notebooks of 
unpublished results. The writings in these notebooks illustrate Ramanujan’s insights but are quite sketchy. Many mathematicians 
have devoted many years of work to explaining and justifying the results in these notebooks. An excellent movie, The Man Who 
Knew Infinity, about the life of Ramanujan was released in 2015. 
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by contrapostion. If neither of these approaches works, you might try a proof by contradiction. 
However, we did not provide any further guidance how to create such proofs. We now present 
some strategies that you can use to develop new proofs. 


FORWARD AND BACKWARD REASONING Whichever method you choose, you need a 
starting point for your proof. To begin a direct proof of a conditional statement, you start with the 
premises. Using these premises, together with axioms and known theorems, you can construct 
a proof using a sequence of steps that leads to the conclusion. This type of reasoning, called 
forward reasoning, is the most common type of reasoning used to prove relatively simple results. 
Similarly, with indirect reasoning you can start with the negation of the conclusion and, using 
a sequence of steps, obtain the negation of the premises. 

Unfortunately, forward reasoning is often difficult to use to prove more complicated results, 
because the reasoning needed to reach the desired conclusion may be far from obvious. In such 
cases it may be helpful to use backward reasoning. To reason backward to prove a statement q, 
we find a statement p that we can prove with the property that p > q. (Note that it is not helpful 
to find a statement r that you can prove such that q > r, because it is the fallacy of begging 
the question to conclude from q > r and r that q is true.) Backward reasoning is illustrated in 
Examples 14 and 15. 


Given two positive real numbers x and y, their arithmetic mean is (x + y)/2 and their geometric 
mean is ,/xy. When we compare the arithmetic and geometric means of pairs of distinct positive 
real numbers, we find that the arithmetic mean is always greater than the geometric mean. [For 


example, when x = 4 and y = 6, we have 5 = (4+ 6)/2 > y4- 6 = y24.] Can we prove that 
this inequality is always true? 


Solution: To prove that (x + y)/2 > ,/xy when x and y are distinct positive real numbers, 
we can work backward. We construct a sequence of equivalent inequalities. The equivalent 
inequalities are 


(x+y)/2 > sry, 
Œ +y)? /4 > xy, 
(x+y)? > 4xy, 
x? + 2xy +y? > 4xy, 
x —2xy+y? > 0, 
(x - yy > 0. 


Because (x — y)? > 0 when x Æ y, it follows that the final inequality is true. Because all these 
inequalities are equivalent, it follows that (x + y)/2 > xy when x Æ y. Once we have carried 
out this backward reasoning, we can build a proof based on reversing the steps. This produces 
construct a proof using forward reasoning. (Note that the steps of our backward reasoning will 
not be part of the final proof. These steps serve as our guide for putting this proof together.) 


Proof: Suppose that x and y are distinct positive real numbers. Then (x —y)* > 0 be- 
cause the square of a nonzero real number is positive (see Appendix 1). Because (x — y}? = 
x? —2xy + y’, this implies that x? — 2xy + y? > 0. Adding 4xy to both sides, we obtain x? + 
2xy + y? > 4xy. Because x? + 2xy + y? = (x + y)’, this means that (x + y)? > 4xy. Dividing both 
sides of this equation by 4, we see that (x + y)?/4 > xy. Finally, taking square roots of both sides 
(which preserves the inequality because both sides are positive) yields (x + y)/2 > y. We 
conclude that if x and y are distinct positive real numbers, then their arithmetic mean (x + y)/2 
is greater than their geometric mean 4/xy. K 


106 


1 / The Foundations: Logic and Proofs 


EXAMPLE 15 


EXAMPLE 16 


Extra 
Examples 


» 


Suppose that two people play a game taking turns removing one, two, or three stones at a time 
from a pile that begins with 15 stones. The person who removes the last stone wins the game. 
Show that the first player can win the game no matter what the second player does. 


Solution: To prove that the first player can always win the game, we work backward. At the 
last step, the first player can win if this player is left with a pile containing one, two, or three 
stones. The second player will be forced to leave one, two, or three stones if this player has to 
remove stones from a pile containing four stones. Consequently, one way for the first person to 
win is to leave four stones for the second player on the next-to-last move. The first person can 
leave four stones when there are five, six, or seven stones left at the beginning of this player’s 
move, which happens when the second player has to remove stones from a pile with eight stones. 
Consequently, to force the second player to leave five, six, or seven stones, the first player should 
leave eight stones for the second player at the second-to-last move for the first player. This means 
that there are nine, ten, or eleven stones when the first player makes this move. Similarly, the 
first player should leave twelve stones when this player makes the first move. We can reverse 
this argument to show that the first player can always make moves so that this player wins 
the game no matter what the second player does. These moves successively leave twelve, eight, 
and four stones for the second player. < 


ADAPTING EXISTING PROOFS An excellent way to look for possible approaches that can 
be used to prove a statement is to take advantage of existing proofs of similar results. Often 
an existing proof can be adapted to prove other facts. Even when this is not the case, some of 
the ideas used in existing proofs may be helpful. Because existing proofs provide clues for new 
proofs, you should read and understand the proofs you encounter in your studies. This process 
is illustrated in Example 16. 


In Example 11 of Section 1.7 we proved that V2 is irrational. We now conjecture that V3 


is irrational. Can we adapt the proof in Example 11 in Section 1.7 to show that V3 is 
irrational? 


Solution: To adapt the proof in Example 11 in Section 1.7, we begin by mimicking the steps in 


that proof, but with V2 replaced with V3 . First, we suppose that V3 = c/d where the fraction 
c/d is in lowest terms. Squaring both sides tells us that 3 = c”/d’, so that 3d” = c*. Can we 
use this equation to show that 3 must be a factor of both c and d, similar to how we used the 
equation 2b? = a” in Example 11 in Section 1.7 to show that 2 must be a factor of both a 
and b? (Recall that an integer s is a factor of the integer t if t/s is an integer. An integer n is 
even if and only if 2 is a factor of n.) In turns out that we can, but we need some ammunition 
from number theory, which we will develop in Chapter 4. We sketch out the remainder of the 
proof, but leave the justification of these steps until Chapter 4. Because 3 is a factor of c°, it 
must also be a factor of c. Furthermore, because 3 is a factor of c, 9 is a factor of c?, which 
means that 9 is a factor of 3d. This implies that 3 is a factor of d*, which means that 3 is a 
factor of that d. This makes 3 a factor of both c and d, which contradicts the assumption that 
c/d is in lowest terms. After we have filled in the justification for these steps, we will have 


shown that y3 is irrational by adapting the proof that V2 is irrational. Note that this proof can 


be extended to show that s/n is irrational whenever n is a positive integer that is not a perfect 
square. We leave the details of this to Chapter 4. < 


A good tip is to look for existing proofs that you might adapt when you are confronted 
with proving a new theorem, particularly when the new theorem seems similar to one you have 
already proved. 
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1.8.6 Looking for Counterexamples 


In Section 1.7 we introduced the use of counterexamples to show that certain statements are 
false. When confronted with a conjecture, you might first try to prove this conjecture, and if 
your attempts are unsuccessful, you might try to find a counterexample, first by looking at 
the simplest, smallest examples. If you cannot find a counterexample, you might again try to 
prove the statement. In any case, looking for counterexamples is an extremely important pursuit, 
which often provides insights into problems. We will illustrate the role of counterexamples in 
Example 17. 


In Example 15 in Section 1.7 we showed that the statement “Every positive integer is the sum of 
two squares of integers” is false by finding a counterexample. That is, there are positive integers 
that cannot be written as the sum of the squares of two integers. Although we cannot write every 
positive integer as the sum of the squares of two integers, maybe we can write every positive 
integer as the sum of the squares of three integers. That is, is the statement “Every positive 
integer is the sum of the squares of three integers” true or false? 


Solution: Because we know that not every positive integer can be written as the sum of two 
squares of integers, we might initially be skeptical that every positive integer can be written as 
the sum of three squares of integers. So, we first look for a counterexample. That is, we can show 
that the statement “Every positive integer is the sum of three squares of integers” is false if we 
can find a particular integer that is not the sum of the squares of three integers. To look for a 
counterexample, we try to write successive positive integers as a sum of three squares. We find 
that 1=0?+0°4+17,2=041774+17,3=1747417,4=040?+27,5=04+17+2?, 
6 = 1? + 1? + 2?, but we cannot find a way to write 7 as the sum of three squares. To show that 
there are not three squares that add up to 7, we note that the only possible squares we can use 
are those not exceeding 7, namely, 0, 1, and 4. Because no three terms where each term is 0, 1, 
or 4 add up to 7, it follows that 7 is a counterexample. We conclude that the statement “Every 
positive integer is the sum of the squares of three integers” is false. 

We have shown that not every positive integer is the sum of the squares of three integers. 
The next question to ask is whether every positive integer is the sum of the squares of four 
positive integers. Some experimentation provides evidence that the answer is yes. For exam- 
ple, 7 = 1? + 1? + 1? +2?, 25 = 4? +2? +2? + 1°, and 87 = 9? + 2? + 1? + 1°. It turns out the 
conjecture “Every positive integer is the sum of the squares of four integers” is true. For a proof, 
see [R010]. < 


1.8.7 Proof Strategy in Action 


Mathematics is generally taught as if mathematical facts were carved in stone. Mathematics 
texts (including the bulk of this book) formally present theorems and their proofs. Such presen- 
tations do not convey the discovery process in mathematics. This process begins with exploring 
concepts and examples, asking questions, formulating conjectures, and attempting to settle these 
conjectures either by proof or by counterexample. These are the day-to-day activities of math- 
ematicians. Believe it or not, the material taught in textbooks was originally developed in this 
way. 

People formulate conjectures on the basis of many types of possible evidence. The exam- 
ination of special cases can lead to a conjecture, as can the identification of possible patterns. 
Altering the hypotheses and conclusions of known theorems also can lead to plausible conjec- 
tures. At other times, conjectures are made based on intuition or a belief that a result holds. 
No matter how a conjecture was made, once it has been formulated, the goal is to prove or dis- 
prove it. When mathematicians believe that a conjecture may be true, they try to find a proof. If 
they cannot find a proof, they may look for a counterexample. When they cannot find a coun- 
terexample, they may switch gears and once again try to prove the conjecture. Although many 
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FIGURE 3 
FIGURE 2 The standard checkerboard. Two dominoes. 


conjectures are quickly settled, a few conjectures resist attack for hundreds of years and lead to 
the development of new parts of mathematics. We will mention a few famous conjectures later 
in this section. 


1.8.8 Tilings 


We can illustrate aspects of proof strategy through a brief study of tilings of checkerboards. 
Looking at tilings of checkerboards is a fruitful way to quickly discover many different results 
and construct their proofs using a variety of proof methods. There are almost an endless number 
of conjectures that can be made and studied in this area, too. To begin, we need to define some 
terms. A checkerboard is a rectangle divided into squares of the same size by horizontal and 
vertical lines. The game of checkers is played on a board with 8 rows and 8 columns; this 
board is called the standard checkerboard and is shown in Figure 2. In this section we use the 
term board to refer to a checkerboard of any rectangular size as well as parts of checkerboards 
obtained by removing one or more squares. A domino is a rectangular piece that is one square 
by two squares, as shown in Figure 3. We say that a board is tiled by dominoes when all its 
squares are covered with no overlapping dominoes and no dominoes overhanging the board. 
We now develop some results about tiling boards using dominoes. 


Can we tile the standard checkerboard using dominoes? 


Solution: We can find many ways to tile the standard checkerboard using dominoes. For exam- 
ple, we can tile it by placing 32 dominoes horizontally, as shown in Figure 4. The existence 
of one such tiling completes a constructive existence proof. There are a large number of other 
ways to do this tiling. We can place 32 dominoes vertically on the board or we can place some 
tiles vertically and some horizontally. But for a constructive existence proof we needed to find 
just one such tiling. < 


Can we tile a board obtained by removing one of the four corner squares of a standard checker- 
board? 


Solution: To answer this question, note that a standard checkerboard has 64 squares, so removing 
a square produces a board with 63 squares. Now suppose that we could tile a board obtained 
from the standard checkerboard by removing a corner square. The board has an even number of 
squares because each domino covers two squares and no two dominoes overlap and no dominoes 
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FIGURE 4 Tiling the standard checkerboard. FIGURE 5 The standard checkerboard 


EXAMPLE 20 


with the upper left and lower right squares 
removed. 


overhang the board. Consequently, we can prove by contradiction that a standard checkerboard 
with one square removed cannot be tiled using dominoes because such a board has an odd 
number of squares. < 


We now consider a trickier situation. 


Can we tile the board obtained by deleting the upper left and lower right corner squares of a 
standard checkerboard, shown in Figure 5? 


Solution: A board obtained by deleting two squares of a standard checkerboard contains 
64 — 2 = 62 squares. Because 62 is even, we cannot quickly rule out the existence of a tiling of 
the standard checkerboard with its upper left and lower right squares removed, unlike Example 
19, where we ruled out the existence of a tiling of the standard checkerboard with one corner 
square removed. Trying to construct a tiling of this board by successively placing dominoes 
might be a first approach, as the reader should attempt. However, no matter how much we try, 
we cannot find such a tiling. Because our efforts do not produce a tiling, we are led to conjecture 
that no tiling exists. 

We might try to prove that no tiling exists by showing that we reach a dead end however 
we successively place dominoes on the board. To construct such a proof, we would have to 
consider all possible cases that arise as we run through all possible choices of successively 
placing dominoes. For example, we have two choices for covering the square in the second 
column of the first row, next to the removed top left corner. We could cover it with a horizontally 
placed tile or a vertically placed tile. Each of these two choices leads to further choices, and so 
on. It does not take long to see that this is not a fruitful plan of attack for a person, although a 
computer could be used to complete such a proof by exhaustion. (Exercise 47 asks you to supply 
such a proof to show that a 4 x 4 checkerboard with opposite corners removed cannot be tiled.) 

We need another approach. Perhaps there is an easier way to prove there is no tiling of 
a standard checkerboard with two opposite corners removed. As with many proofs, a key ob- 
servation can help. We color the squares of this checkerboard using alternating white and black 
squares, as in Figure 2. Observe that a domino in a tiling of such a board covers one white square 
and one black square. Next, note that this board has unequal numbers of white squares and black 
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FIGURE 6 A 
right triomino 
and a straight 

triomino. 


EXAMPLE 21 


EXAMPLE 22 


squares. We can use these observations to prove by contradiction that a standard checkerboard 
with opposite corners removed cannot be tiled using dominoes. We now present such a proof. 


Proof: Suppose we can use dominoes to tile a standard checkerboard with opposite cor- 
ners removed. Note that the standard checkerboard with opposite corners removed contains 
64 — 2 = 62 squares. The tiling would use 62/2 = 31 dominoes. Note that each domino in this 
tiling covers one white and one black square. Consequently, the tiling covers 31 white squares 
and 31 black squares. However, when we remove two opposite corner squares, either 32 of the 
remaining squares are white and 30 are black or else 30 are white and 32 are black. This contra- 
dicts the assumption that we can use dominoes to cover a standard checkerboard with opposite 
corners removed, completing the proof. < 


We can use other types of pieces besides dominoes in tilings. Instead of dominoes we can 
study tilings that use identically shaped pieces constructed from congruent squares that are 
connected along their edges. Such pieces are called polyominoes, a term coined in 1953 by the 
mathematician Solomon Golomb, the author of an entertaining book about them [Go94]. We 
will consider two polyominoes with the same number of squares the same if we can rotate and/or 
flip one of the polyominoes to get the other one. For example, there are two types of triominoes 
(see Figure 6), which are polyominoes made up of three squares connected by their sides. One 
type of triomino, the straight triomino, has three horizontally connected squares; the other 
type, right triominoes, resembles the letter L in shape, flipped and/or rotated, if necessary. We 
will study the tilings of a checkerboard by straight triominoes here; we will study tilings by right 
triominoes in Section 5.1. 


Can you use straight triominoes to tile a standard checkerboard? 


Solution: The standard checkerboard contains 64 squares and each triomino covers three 
squares. Consequently, if triominoes tile a board, the number of squares of the board must be 
a multiple of 3. Because 64 is not a multiple of 3, triominoes cannot be used to cover an 8 x 8 
checkerboard. < 


In Example 22, we consider the problem of using straight triominoes to tile a standard 
checkerboard with one corner missing. 


Can we use straight triominoes to tile a standard checkerboard with one of its four corners 
removed? An 8 x 8 checkerboard with one corner removed contains 64 — 1 = 63 squares. Any 
tiling by straight triominoes of one of these four boards uses 63/3 = 21 triominoes. However, 
when we experiment, we cannot find a tiling of one of these boards using straight triominoes. 
A proof by exhaustion does not appear promising. Can we adapt our proof from Example 20 to 
prove that no such tiling exists? 


Solution: We will color the squares of the checkerboard in an attempt to adapt the proof by 
contradiction we gave in Example 20 of the impossibility of using dominoes to tile a standard 
checkerboard with opposite corners removed. Because we are using straight triominoes rather 
than dominoes, we color the squares using three colors rather than two colors, as shown in Fig- 
ure 7. Note that there are 21 blue squares, 21 black squares, and 22 white squares in this coloring. 
Next, we make the crucial observation that when a straight triomino covers three squares of the 
checkerboard, it covers one blue square, one black square, and one white square. Next, note that 
each of the three colors appears in a corner square. Thus, without loss of generality, we may 
assume that we have rotated the coloring so that the missing square is colored blue. Therefore, 
we assume that the remaining board contains 20 blue squares, 21 black squares, and 22 white 
squares. 
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FIGURE7_ Coloring the squares of the standard checkerboard 
with three colors. 


If we could tile this board using straight triominoes, then we would use 63/3 = 21 straight 
triominoes. These triominoes would cover 21 blue squares, 21 black squares, and 21 white 
squares. This contradicts the fact that this board contains 20 blue squares, 21 black squares, and 
22 white squares. Therefore, we cannot tile this board using straight triominoes. < 


1.8.9 The Role of Open Problems 


Many advances in mathematics have been made by people trying to solve famous unsolved 
problems. In the past 20 years, many unsolved problems have finally been resolved, such as the 
proof of a conjecture in number theory made more than 300 years ago. This conjecture asserts 
the truth of the statement known as Fermat’s last theorem. 


FERMAT’S LAST THEOREM The equation 
x +y” = Z 


has no solutions in integers x, y, and z with xyz 4 0 whenever n is an integer with n > 2. 


Remark: The equation x? + y? = z? has infinitely many solutions in integers x, y, and z; these 
solutions are called Pythagorean triples and correspond to the lengths of the sides of right tri- 
angles with integer lengths. See Exercise 34. 


This problem has a fascinating history. In the seventeenth century, Fermat jotted in the mar- 
gin of his copy of the works of Diophantus that he had a “wondrous proof” that there are no 
integer solutions of x” + y” = z” when n is an integer greater than 2 with xyz # 0. However, 
he never published a proof (Fermat published almost nothing), and no proof could be found in 
the papers he left when he died. Mathematicians looked for a proof for three centuries without 
success, although many people were convinced that a relatively simple proof could be found. 
(Proofs of special cases were found, such as the proof of the case when n = 3 by Euler and 
the proof of the n = 4 case by Fermat himself.) Over the years, several established mathemati- 
cians thought that they had proved this theorem. In the nineteenth century, one of these failed 
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attempts led to the development of the part of number theory called algebraic number theory. 
A correct proof, requiring hundreds of pages of advanced mathematics, was not found until the 
1990s, when Andrew Wiles used recently developed ideas from a sophisticated area of number 
theory called the theory of elliptic curves to prove Fermat’s last theorem. Wiles’s quest to find 
a proof of Fermat’s last theorem using this powerful theory, described in a program in the Nova 
series on public television, took close to ten years! Moreover, his proof was based on major 
contributions of many mathematicians. (The interested reader should consult [Ro10] for more 
information about Fermat’s last theorem and for additional references concerning this problem 
and its resolution.) 

We now state an open problem that is simple to describe, but that seems quite difficult to 
resolve. 


EXAMPLE 23 The 3x +1 Conjecture Let T be the transformation that sends an even integer x to x/2 and 

an odd integer x to 3x + 1. A famous conjecture, sometimes known as the 3x + 1 conjecture, 

Links » states that for all positive integers x, when we repeatedly apply the transformation T, we 

will eventually reach the integer 1. For example, starting with x = 13, we find 7(13) = 

3-13+1=40, 7T(40)=40/2=20, 7(20)=20/2=10, 7(10)=10/2=5, T(5)= 

3-54+1=16, T(16)=8, 7(8)=4, 7T(4) =2, and T(2)= 1. The 3x+ 1 conjecture has 
been verified using computers for all integers x up to 5.48 - 10!8. 

The 3x + 1 conjecture has an interesting history and has attracted the attention of mathe- 
maticians since the 1950s. The conjecture has been raised many times and goes by many other 
names, including the Collatz problem, Hasse’s algorithm, Ulam’s problem, the Syracuse prob- 
lem, and Kakutani’s problem. Many mathematicians have been diverted from their work to 
spend time attacking this conjecture. This led to the joke that this problem was part of a conspir- 
acy to slow down American mathematical research. See the article by Jeffrey Lagarias [La10] 


wadde for a fascinating discussion of this problem and the results that have been found by mathemati- 


cians attacking it. < 


There are a surprising number of important open problems throughout discrete mathemat- 
ics. For instance, in Chapter 4 you will encounter many open questions about prime numbers. 
(Students already familiar with the basic notions about primes might want to explore Section 
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sor of Divinity. Wiles attended the King’s College School and the Leys School in Cambridge. Wiles become 
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he would never let this problem go, as it looked simple but none of the great mathematicians could solve it. 
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theory of elliptic curves. He was a Benjamin Peirce Assistant Professor at Harvard University from 1977 
£ until 1980. In 1981, he held a post at the Institute for Advanced Study in Princeton, and in 1982 he was 
©Charles Rex ArbogasVAP_ appointed to a professorship at Princeton University. He was awarded a Guggenheim Fellowship in 1985 
images and spent a year at the Institut des Hautes Études Scientifiques and the École Normale Supérieure in Paris. 
Ironically, he did not realize that during his years working on elliptic curves he was learning techniques that would later help him 
solve the problem that obsessed him. 

In 1986 when Wiles learned of work that showed that Fermat’s last theorem follows from a conjecture in the theory of elliptic 
curves, he realized that this led to a possible strategy for a poof. He abandoned his ongoing research and devoted himself entirely to 
working on Fermat’s last theorem. It took him more than seven years to complete his proof and two more years for some parts of the 
proof to be corrected. During this time he spent time only on this problem and with his young daughters. In 1988 he took a position 
as a research professor at Oxford University, returning to Princeton in 1990, where he remained until 2011, when he rejoined Oxford 
University as the Royal Society Research Professor. 

Not only did Wiles become famous when he proved Fermat’s last theorem, he also won the Wolfskehl Prize, which was es- 
tablished in 1908 for the first correct proof. This prize included 100,000 German marks (in the currency of the day), which would 
have been worth over $1,500,000 today. Although it was common wisdom that this prize had become worthless because of the two 
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4.3, where these open questions are discussed.) You will encounter many other open questions 
as you read this book. The study of such problems has played and continues to play an important 
role in the development of many parts of discrete mathematics. 


1.8.10 Additional Proof Methods 


In this chapter we introduced the basic methods used in proofs. We also described how to lever- 
age these methods to prove a variety of results. We will use these proof methods in all subse- 
quent chapters. In particular, we will use them in Chapters 2, 3, and 4 to prove results about 
sets, functions, algorithms, and number theory and in Chapters 9, 10, and 11 to prove results in 
graph theory. Among the theorems we will prove is the famous halting theorem, which states 
that there is a problem that cannot be solved using any procedure. However, there are many im- 
portant proof methods besides those we have covered. We will introduce some of these methods 
later in this book. In particular, in Section 5.1 we will discuss mathematical induction, which 
is an extremely useful method for proving statements of the form VnP(n), where the domain 
consists of all positive integers. In Section 5.3 we will introduce structural induction, which can 
be used to prove results about recursively defined sets. We will use the Cantor diagonalization 
method, which can be used to prove results about the size of infinite sets, in Section 2.5. In 
Chapter 6 we will introduce the notion of combinatorial proofs, which can be used to prove re- 
sults by counting arguments. The reader should note that entire books have been devoted to the 
activities discussed in this section, including many excellent works by George Pólya ([Po61], 
[Po71], [Po90]). 

Finally, note that we have not given a procedure that can be used for proving theorems in 
mathematics. It is a deep theorem of mathematical logic that there is no such procedure. 


1. Prove that n? + 1 > 2” when n is a positive integer with 9. Prove the triangle inequality, which states that if x and y 


láns 


2. Use a proof by cases to show that 10 is not the square of a 
positive integer. [Hint: Consider two cases: (i) 1 < x < 3, 


(ii) x > 4] 


are real numbers, then |x| + |y| > |x + y| (where |x| rep- 
resents the absolute value of x, which equals x if x > 0 
and equals —x if x < 0). 

10. Prove that there is a positive integer that equals the sum 
of the positive integers not exceeding it. Is your proof 


3. Use a proof by cases to show that 100 is not the cube of a 


positive integer. [Hint: Consider two cases: (i) 1 < x < 4, 
(ii) x > 5.] 


. Prove that there are no positive perfect cubes less than 
1000 that are the sum of the cubes of two positive 
integers. 


. Prove that if x and y are real numbers, then max(x, y) + 
min(x, y) = x + y. [Hint: Use a proof by cases, with the 
two cases corresponding to x> y and x< y, respec- 
tively. ] 


. Use a proof by cases to show that min(a, min(b, c)) = 
min(min(a, b), c) whenever a, b, and c are real numbers. 


. Prove using the notion of without loss of generality that 
min(x, y) = (x + y — |x — y|)/2 and max(x, y) = (x + y + 
|x — y|)/2 whenever x and y are real numbers. 


. Prove using the notion of without loss of generality that 
5x + 5y is an odd integer when x and y are integers of 


opposite parity. 


11. 


12. 


13. 


14. 


15. 


16. 


constructive or nonconstructive? 

Prove that there are 100 consecutive positive integers that 
are not perfect squares. Is your proof constructive or non- 
constructive? 

Prove that either 2 - 105% + 15 or 2 - 105% + 16 is not a 
perfect square. Is your proof constructive or nonconstruc- 
tive? 

Prove that there exists a pair of consecutive integers such 
that one of these integers is a perfect square and the other 
is a perfect cube. 

Show that the product of two of the numbers 65! — 
92001 Ş 3177, 79!2!2 _ 92399 fe 22001 | and 944493 = 58192 ue 
7'777 is nonnegative. Is your proof constructive or non- 
constructive? [Hint: Do not try to evaluate these num- 
bers! ] 

Prove or disprove that there is a rational number x and an 
irrational number y such that x’ is irrational. 

Prove or disprove that if a and b are rational numbers, 
then a? is also rational. 
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Show that each of these statements can be used to ex- 
press the fact that there is a unique element x such that 
P(x) is true. [Note that we can also write this statement 
as A!xP(x).] 

a) AxVy(P(y) e x = y) 

b) AxP(x) A VxVy(P(x) A PO) > x = y) 

©) Ax(P(x) A Vy(PQ) > x = y)) 

Show that if a, b, and c are real numbers and a # 0, then 
there is a unique solution of the equation ax + b = c. 


Suppose that a and b are odd integers with a 4 b. Show 
there is a unique integer c such that |a — c| = |b — cl. 


Show that if r is an irrational number, there is a unique 
integer n such that the distance between r and n is less 
than 1/2. 

Show that if n is an odd integer, then there is a unique 
integer k such that n is the sum of k — 2 and k + 3. 
Prove that given a real number x there exist unique num- 


bers n and e such that x =n +e, n is an integer, and 
O<e<l. 


Prove that given a real number x there exist unique num- 
bers n and e such that x =n-— e€, n is an integer, and 
O<e<l. 


Use forward reasoning to show that if x is a nonzero 
real number, then x? + 1 /x° > 2. [Hint: Start with the in- 
equality (x — 1/x)? > 0, which holds for all nonzero real 
numbers x. ] 


The harmonic mean of two real numbers x and y equals 
2xy/(x + y). By computing the harmonic and geometric 
means of different pairs of positive real numbers, formu- 
late a conjecture about their relative sizes and prove your 
conjecture. 


The quadratic mean of two real numbers x and y 
equals y(x? + y*)/2. By computing the arithmetic and 
quadratic means of different pairs of positive real num- 
bers, formulate a conjecture about their relative sizes and 
prove your conjecture. 


Write the numbers 1, 2, ..., 2n on a blackboard, where 
n is an odd integer. Pick any two of the numbers, j and 
k, write |j — k| on the board and erase j and k. Continue 
this process until only one integer is written on the board. 
Prove that this integer must be odd. 

Suppose that five ones and four zeros are arranged around 
a circle. Between any two equal bits you insert a 0 and be- 
tween any two unequal bits you insert a 1 to produce nine 
new bits. Then you erase the nine original bits. Show that 
when you iterate this procedure, you can never get nine 
zeros. [Hint: Work backward, assuming that you did end 
up with nine zeros. ] 

Formulate a conjecture about the decimal digits that ap- 
pear as the final decimal digit of the fourth power of an 
integer. Prove your conjecture using a proof by cases. 
Formulate a conjecture about the final two decimal digits 
of the square of an integer. Prove your conjecture using a 
proof by cases. 

Prove that there is no positive integer n such that n? + 
n? = 100. 


32. 


33. 


34. 


35. 


36. 
37. 


38. 


* 39, 


40. 


41. 


42. 


43. 


44. 


45. 


46. 


47. 


Prove that there are no solutions in integers x and y to the 
equation 2x? + 5y? = 14. 

Prove that there are no solutions in positive integers x and 
y to the equation x* + y* = 625. 

Prove that there are infinitely many solutions in pos- 
itive integers x, y, and z to the equation e+ y = 
2. [Hint: Let x = m —n?, y = 2mn, and z= m +n’, 
where m and n are integers. | 

Adapt the proof in Example 4 in Section 1.7 to prove that 
if n = abc, where a, b, and c are positive integers, then 


a < ẹ/n,b < ẹ&/n, or c < Wn. 


Prove that 1/2 is irrational. 


Prove that between every two rational numbers there is 
an irrational number. 


Prove that between every rational number and every irra- 

tional number there is an irrational number. 

Let S = x,y, HX2y2 + +++ + X Yn Where x}, X2, .-., x, and 

Yp Yo +++, Yn are orderings of two different sequences of 

positive real numbers, each containing n elements. 

a) Show that S takes its maximum value over all order- 
ings of the two sequences when both sequences are 
sorted (so that the elements in each sequence are in 
nondecreasing order). 

b) Show that S takes its minimum value over all or- 
derings of the two sequences when one sequence 
is sorted into nondecreasing order and the other is 
sorted into nonincreasing order. 

Prove or disprove that if you have an 8-gallon jug of wa- 

ter and two empty jugs with capacities of 5 gallons and 3 

gallons, respectively, then you can measure 4 gallons by 

successively pouring some of or all of the water in a jug 
into another jug. 


Verify the 3x + 1 conjecture for these integers. 

a) 6 b) 7 c) 17 d) 21 

Verify the 3x + 1 conjecture for these integers. 

a) 16 b) 11 c) 35 d) 113 

Prove or disprove that you can use dominoes to tile 
the standard checkerboard with two adjacent corners re- 
moved (that is, corners that are not opposite). 


Prove or disprove that you can use dominoes to tile a stan- 
dard checkerboard with all four corners removed. 


Prove that you can use dominoes to tile a rectangular 
checkerboard with an even number of squares. 


Prove or disprove that you can use dominoes to tile a 
5 x 5 checkerboard with three corners removed. 


Use a proof by exhaustion to show that a tiling using 
dominoes of a 4 X 4 checkerboard with opposite corners 
removed does not exist. [Hint: First show that you can 
assume that the squares in the upper left and lower right 
corners are removed. Number the squares of the original 
checkerboard from 1 to 16, starting in the first row, mov- 
ing right in this row, then starting in the leftmost square 
in the second row and moving right, and so on. Remove 
squares 1 and 16. To begin the proof, note that square 
2 is covered either by a domino laid horizontally, which 


covers squares 2 and 3, or vertically, which covers squares 
2 and 6. Consider each of these cases separately, and work 
through all the subcases that arise. ] 

*48. Prove that when a white square and a black square are 
removed from an 8 x 8 checkerboard (colored as in the 
text) you can tile the remaining squares of the checker- 
board using dominoes. [Hint: Show that when one black 
and one white square are removed, each part of the par- 
tition of the remaining cells formed by inserting the bar- 
riers shown in the figure can be covered by dominoes.] 


Key Terms and Results 


TERMS 


proposition: a statement that is true or false 

propositional variable: a variable that represents a proposi- 
tion 

truth value: true or false 


— p (negation of p): the proposition with truth value opposite 
to the truth value of p 

logical operators: operators used to combine propositions 

compound proposition: a proposition constructed by combin- 
ing propositions using logical operators 

truth table: a table displaying all possible truth values of 
propositions 

p V q (disjunction of p and q): the proposition “p or q,” which 
is true if and only if at least one of p and q is true 

p ^ q (conjunction of p and q): the proposition “p and q,” 
which is true if and only if both p and q are true 


p ® q (exclusive or of p and q): the proposition “p XOR q,” 
which is true when exactly one of p and q is true 

p > q (p implies q): the proposition “if p, then q,” which is 
false if and only if p is true and q is false 

converse of p — q: the conditional statement q > p 

contrapositive of p — q: the conditional statement 7q > =p 

inverse of p — q: the conditional statement =p > 7g 

p  q (biconditional): the proposition “p if and only if q,” 
which is true if and only if p and q have the same truth 
value 


bit: either a 0 ora 1 


Boolean variable: a variable that has a value of 0 or 1 
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49. Show that by removing two white squares and two black 
squares from an 8 X 8 checkerboard (colored as in the 
text) you can make it impossible to tile the remaining 
squares using dominoes. 


* 50. Find all squares, if they exist, on an 8 x 8 checkerboard 
such that the board obtained by removing one of these 
squares can be tiled using straight triominoes. [Hint: First 
use arguments based on coloring and rotations to elimi- 
nate as many squares as possible from consideration. ] 


* 51. a) Draw each of the five different tetrominoes, where a 
tetromino is a polyomino consisting of four squares. 


b) For each of the five different tetrominoes, prove or 
disprove that you can tile a standard checkerboard us- 
ing these tetrominoes. 


* 52. Prove or disprove that you can tile a 10 x 10 checker- 
board using straight tetrominoes. 


bit operation: an operation on a bit or bits 

bit string: a list of bits 

bitwise operations: operations on bit strings that operate on 
each bit in one string and the corresponding bit in the other 
string 

logic gate: a logic element that performs a logical operation 
on one or more bits to produce an output bit 


logic circuit: a switching circuit made up of logic gates that 
produces one or more output bits 


tautology: a compound proposition that is always true 
contradiction: a compound proposition that is always false 


contingency: a compound proposition that is sometimes true 
and sometimes false 


consistent compound propositions: compound propositions 
for which there is an assignment of truth values to the vari- 
ables that makes all these propositions true 


satisfiable compound proposition: a compound proposition 
for which there is an assignment of truth values to its vari- 
ables that makes it true 


logically equivalent compound propositions: compound 
propositions that always have the same truth values 


predicate: part of a sentence that attributes a property to the 
subject 


propositional function: a statement containing one or more 
variables that becomes a proposition when each of its vari- 
ables is assigned a value or is bound by a quantifier 


domain (or universe) of discourse: the values a variable in a 
propositional function may take 
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dx P(x) (existential quantification of P(x)): the proposition 
that is true if and only if there exists an x in the domain 
such that P(x) is true 

VxP(x) (universal quantification of P(x)): the proposition 
that is true if and only if P(x) is true for every x in the do- 
main 

logically equivalent expressions: expressions that have the 
same truth value no matter which propositional functions 
and domains are used 

free variable: a variable not bound in a propositional function 

bound variable: a variable that is quantified 

scope of a quantifier: portion of a statement where the quan- 
tifier binds its variable 

argument: a sequence of statements 

argument form: a sequence of compound propositions in- 
volving propositional variables 

premise: a statement, in an argument, or argument form, other 
than the final one 

conclusion: the final statement in an argument or argument 
form 

valid argument form: a sequence of compound propositions 
involving propositional variables where the truth of all the 
premises implies the truth of the conclusion 

valid argument: an argument with a valid argument form 

rule of inference: a valid argument form that can be used in 
the demonstration that arguments are valid 

fallacy: an invalid argument form often used incorrectly as a 
rule of inference (or sometimes, more generally, an incor- 
rect argument) 

circular reasoning or begging the question: reasoning where 
one or more steps are based on the truth of the statement 
being proved 

theorem: a mathematical assertion that can be shown to be 
true 

conjecture: a mathematical assertion proposed to be true, but 
that has not been proved 

proof: a demonstration that a theorem is true 

axiom: a statement that is assumed to be true and that can be 
used as a basis for proving theorems 


Review Questions 


1. a) Define the negation of a proposition. 


b) What is the negation of “This is a boring course”? 


2. a) Define (using truth tables) the disjunction, conjunc- 
tion, exclusive or, conditional, and biconditional of 
the propositions p and q. 

b) What are the disjunction, conjunction, exclusive or, 
conditional, and biconditional of the propositions “T 11 
go to the movies tonight” and “T’ll finish my discrete 
mathematics homework”? 


3. a) Describe at least five different ways to write the con- 
ditional statement p —> q in English. 


lemma: a theorem used to prove other theorems 

corollary: a proposition that can be proved as a consequence 
of a theorem that has just been proved 

vacuous proof: a proof that p > q is true based on the fact 
that p is false 

trivial proof: a proof that p > q is true based on the fact that 
q is true 

direct proof: a proof that p > q is true that proceeds by show- 
ing that q must be true when p is true 

proof by contraposition: a proof that p — q is true that pro- 
ceeds by showing that p must be false when q is false 

proof by contradiction: a proof that p is true based on the 
truth of the conditional statement =p —> q, where q is a con- 
tradiction 

exhaustive proof: a proof that establishes a result by checking 
a list of all possible cases 

proof by cases: a proof broken into separate cases, where these 
cases cover all possibilities 

without loss of generality: an assumption in a proof that 
makes it possible to prove a theorem by reducing the num- 
ber of cases to consider in the proof 

counterexample: an element x such that P(x) is false 

constructive existence proof: a proof that an element with a 
specified property exists that explicitly finds such an ele- 
ment 

nonconstructive existence proof: a proof that an element with 
a specified property exists that does not explicitly find such 
an element 

rational number: a number that can be expressed as the ratio 
of two integers p and q such that q # 0 

uniqueness proof: a proof that there is exactly one element 
satisfying a specified property 


RESULTS 


The logical equivalences given in Tables 6, 7, and 8 in Sec- 
tion 1.3. 

De Morgan’s laws for quantifiers. 

Rules of inference for propositional calculus. 

Rules of inference for quantified statements. 


b) Define the converse and contrapositive of a condi- 
tional statement. 

c) State the converse and the contrapositive of the con- 
ditional statement “If it is sunny tomorrow, then I will 
go for a walk in the woods.” 


4. a) What does it mean for two propositions to be logically 
equivalent? 
b) Describe the different ways to show that two com- 
pound propositions are logically equivalent. 
c) Show in at least two different ways that the com- 
pound propositions =p V (r > =q) and =p V ^q V ar 
are equivalent. 


10. 


. (Depends on the Exercise Set in Section 1.3) 


a) Given atruth table, explain how to use disjunctive nor- 
mal form to construct a compound proposition with 
this truth table. 

b) Explain why part (a) shows that the operators A, V, 
and ~ are functionally complete. 

c) Is there an operator such that the set containing just 
this operator is functionally complete? 


. What are the universal and existential quantifications of 


a predicate P(x)? What are their negations? 


. a) What is the difference between the quantification 


AxVyP(x, y) and VyaxP(%, y), where P(x, y) is a predi- 
cate? 

b) Give an example of a predicate P(x,y) such that 
AxVyP(x, y) and VydxP(%, y) have different truth 
values. 


. Describe what is meant by a valid argument in proposi- 


tional logic and show that the argument “If the earth is 
flat, then you can sail off the edge of the earth,” “You can- 
not sail off the edge of the earth,” therefore, “The earth is 
not flat” is a valid argument. 


. Use rules of inference to show that if the premises “All 


zebras have stripes” and “Mark is a zebra” are true, then 

the conclusion “Mark has stripes” is true. 

a) Describe what is meant by a direct proof, a proof by 
contraposition, and a proof by contradiction of a con- 
ditional statement p > q. 


Supplementary Exercises 


1. 


Let p be the proposition “I will do every exercise in 
this book” and q be the proposition “I will get an A in 
this course.” Express each of these as a combination of 
p and q. 


a) I will get an A in this course only if I do every exercise 
in this book. 

b) I will get an A in this course and I will do every exer- 
cise in this book. 

c) Either I will not get an A in this course or I will not 
do every exercise in this book. 

d) For me to get an A in this course it is necessary and 
sufficient that I do every exercise in this book. 


. Find the truth table of the compound proposition (p V 


g) > (p Aar). 


. Show that these compound propositions are tautologies. 


a) Cq ^ (p > 4) > 7p 
b) (pV gq) Ap) > 4 


. Give the converse, the contrapositive, and the inverse of 


these conditional statements. 


a) If it rains today, then I will drive to work. 
b) If |x| = x, then x > 0. 
c) If nis greater than 3, then n? is greater than 9. 


. Given a conditional statement p — q, find the converse of 


its inverse, the converse of its converse, and the converse 
of its contrapositive. 


11. 


12. 


13. 


14. 


15. 


16. 
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b) Give a direct proof, a proof by contraposition, and a 
proof by contradiction of the statement: “If n is even, 
then n + 4 is even.” 


a) Describe a way to prove the biconditional p © q. 


b) Prove the statement: “The integer 3n + 2 is odd if 
and only if the integer 9n + 5 is even, where n is an 
integer.” 


To prove that the statements p,, p2, p3, and p4 are equiva- 
lent, is it sufficient to show that the conditional statements 
P4 > Po, P3 > Py, and p; > pp are valid? If not, provide 
another collection of conditional statements that can be 
used to show that the four statements are equivalent. 


a) Suppose that a statement of the form VxP(x) is false. 
How can this be proved? 


b) Show that the statement “For every positive integer n, 
n? > 2n” is false. 


What is the difference between a constructive and non- 
constructive existence proof? Give an example of each. 


What are the elements of a proof that there is a unique 
element x such that P(x), where P(x) is a propositional 
function? 


Explain how a proof by cases can be used to prove a result 
about absolute values, such as the fact that |xy| = |x||y| 
for all real numbers x and y. 


. Given a conditional statement p > 4q, find the inverse of 


its inverse, the inverse of its converse, and the inverse of 
its contrapositive. 


. Find acompound proposition involving the propositional 


variables p, q, r, and s that is true when exactly three of 
these propositional variables are true and is false other- 
wise. 


. Show that these statements are inconsistent: “If Sergei 


takes the job offer, then he will get a signing bonus.” “If 
Sergei takes the job offer, then he will receive a higher 
salary.” “If Sergei gets a signing bonus, then he will not 
receive a higher salary.” “Sergei takes the job offer.” 


. Show that these statements are inconsistent: “If Miranda 


does not take a course in discrete mathematics, then she 
will not graduate.” “If Miranda does not graduate, then 
she is not qualified for the job.” “If Miranda reads this 
book, then she is qualified for the job.” “Miranda does 
not take a course in discrete mathematics but she reads 
this book.” 


Teachers in the Middle Ages supposedly tested the realtime 
propositional logic ability of a student via a technique known 
as an obligato game. In an obligato game, a number of rounds 
is set and in each round the teacher gives the student succes- 
sive assertions that the student must either accept or reject as 
they are given. When the student accepts an assertion, it is 


118 


1 / The Foundations: Logic and Proofs 


added as a commitment; when the student rejects an assertion 
its negation is added as a commitment. The student passes 
the test if the consistency of all commitments is maintained 
throughout the test. 


10. 


11. 


12. 


Suppose that in a three-round obligato game, the teacher 
first gives the student the proposition p > q, then the 
proposition ~(p V r) V q, and finally, the proposition q. 
For which of the eight possible sequences of three an- 
swers will the student pass the test? 

Suppose that in a four-round obligato game, the teacher 
first gives the student the proposition ~(p > (q A^ r)), then 
the proposition p V ~q, then the proposition =r, and fi- 
nally, the proposition (p A r) V (q > p). For which of the 
16 possible sequences of four answers will the student 
pass the test? 

Explain why every obligato game has a winning strategy. 


Exercises 13 and 14 are set on the island of knights and knaves 
described in Example 7 in Section 1.2. 


13. 


14. 


15. 


16. 


17. 


18. 


Suppose that you meet three people, Aaron, Bohan, and 
Crystal. Can you determine what Aaron, Bohan, and 
Crystal are if Aaron says “All of us are knaves” and Bohan 
says “Exactly one of us is a knave”? 


Suppose that you meet three people, Anita, Boris, and 
Carmen. What are Anita, Boris, and Carmen if Anita 
says “I am a knave and Boris is a knight” and Boris says 
“Exactly one of the three of us is a knight”? 


(Adapted from [Sm78]) Suppose that on an island there 
are three types of people, knights, knaves, and normals 
(also known as spies). Knights always tell the truth, 
knaves always lie, and normals sometimes lie and some- 
times tell the truth. Detectives questioned three inhabi- 
tants of the island—Amy, Brenda, and Claire—as part of 
the investigation of a crime. The detectives knew that one 
of the three committed the crime, but not which one. They 
also knew that the criminal was a knight, and that the 
other two were not. Additionally, the detectives recorded 
these statements: Amy: “I am innocent.” Brenda: “What 
Amy says is true.” Claire: “Brenda is not a normal.” Af- 
ter analyzing their information, the detectives positively 
identified the guilty party. Who was it? 

Show that if S is a proposition, where S is the condi- 
tional statement “If S is true, then unicorns live,” then 
“Unicorns live” is true. Show that it follows that S can- 
not be a proposition. (This paradox is known as Léb’s 
paradox.) 

Show that the argument with premises “The tooth fairy 
is a real person” and “The tooth fairy is not a real per- 
son” and conclusion “You can find gold at the end of the 
rainbow” is a valid argument. Does this show that the 
conclusion is true? 

Suppose that the truth value of the proposition p, is T 
whenever i is an odd positive integer and is F when- 
ever i is an even positive integer. Find the truth values 
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20. 


21. 


22. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


Model 16 x 16 Sudoku puzzles (with 4 x 4 blocks) as 

satisfiability problems. 

Let P(x) be the statement “Student x knows calculus” and 

let Q(y) be the statement “Class y contains a student who 

knows calculus.” Express each of these as quantifications 

of P(x) and QO). 

a) Some students know calculus. 

b) Not every student knows calculus. 

c) Every class has a student in it who knows calculus. 

d) Every student in every class knows calculus. 

e) There is at least one class with no students who know 
calculus. 

Let P(m, n) be the statement “m divides n,” where the do- 

main for both variables consists of all positive integers. 

(By “m divides n” we mean that n = km for some integer 

k.) Determine the truth values of each of these statements. 


a) P(4,5) b) P(2, 4) 
c) VmVn P(m, n) d) 3m Yn P(m, n) 
e) 3nYmP(m, n) f) Vn PC, n) 


Find a domain for the quantifiers in 3x3yx Æ y ^ 
Vz((z = x) V (z = y))) such that this statement is true. 
Find a domain for the quantifiers in 3x3y(x Æ y A^ 
Yz((z=x)v(z=y)) such that this statement is 
false. 

Use existential and universal quantifiers to express the 
statement “No one has more than three grandmothers” 
using the propositional function G(x, y), which represents 
“x is the grandmother of y.” 

Use existential and universal quantifiers to express the 
statement “Everyone has exactly two biological parents” 
using the propositional function P(x, y), which represents 
“x is the biological parent of y.” 

The quantifier 3„ denotes “there exists exactly n,” so that 
4,,xP(x) means there exist exactly n values in the domain 
such that P(x) is true. Determine the true value of these 
statements where the domain consists of all real num- 
bers. 

a) Ix? = —1) b) 4,x(|x| = 0) 

c0) Ixe = 2) d) 4,x(x = |x|) 

Express each of these statements using existential and 
universal quantifiers and propositional logic, where 3, is 
defined in Exercise 26. 

a) AyxP(x) b) 4,xP(x) 

ce) d,xP(x) d) 43xP(x) 

Let P(x,y) be a propositional function. Show that 
dx Vy P(x, y) > Vy dx P(x, y) is a tautology. 

Let P(x) and Q(x) be propositional functions. Show that 
dx (P(x) > Q(x)) and Vx P(x) > 3x Q(x) always have the 
same truth value. 

If Vy dx P(, y) is true, does it necessarily follow that 
dx Vy P(x, y) is true? 

If Vx dy P(x, y) is true, does it necessarily follow that 
dx Vy P(x, y) is true? 


32. 


33. 


34. 


35. 


36. 


37. 


Find the negations of these statements. 


a) If it snows today, then I will go skiing tomorrow. 

b) Every person in this class understands mathematical 
induction. 

c) Some students in this class do not like discrete math- 
ematics. 

d) Inevery mathematics class there is some student who 
falls asleep during lectures. 

Express this statement using quantifiers: “Every student 

in this class has taken some course in every department 

in the school of mathematical sciences.” 


Express this statement using quantifiers: “There is a 
building on the campus of some college in the United 
States in which every room is painted white.” 


Express the statement “There is exactly one student in 
this class who has taken exactly one mathematics class 
at this school” using the uniqueness quantifier. Then ex- 
press this statement using quantifiers, without using the 
uniqueness quantifier. 


Describe a rule of inference that can be used to prove that 
there are exactly two elements x and y in a domain such 
that P(x) and P(y) are true. Express this rule of inference 
as a statement in English. 


Use rules of inference to show that if the premises 
Vx(P(x) > O(x)), Vx(O() > R(x)), and aR(a), where a 


Computer Projects 


Write programs with the specified input and output. 


*3, 


. Given the truth values of the propositions p and q, find the 


truth values of the conjunction, disjunction, exclusive or, 
conditional statement, and biconditional of these proposi- 
tions. 


. Given two bit strings of length n, find the bitwise AND, 


bitwise OR, and bitwise XOR of these strings. 


Give a compound proposition, determine whether it is sat- 
isfiable by checking its truth value for all positive assign- 
ments of truth values to its propositional variables. 


Computations and Explorations 


38. 
39. 


40. 


41. 


42. 


43. 


44. 


45. 


46. 


£S; 
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is in the domain, are true, then the conclusion =P(a) is 
true. 


Prove that if x? is irrational, then x is irrational. 


Prove or disprove that if x? is irrational, then x° is irra- 
tional. 


Prove that given a nonnegative integer n, there is a unique 
nonnegative integer m such that m? < n < (m + 1). 


Prove that there exists an integer m such that m? > 101%, 


Is your proof constructive or nonconstructive? 


Prove that there is a positive integer that can be written 
as the sum of squares of positive integers in two differ- 
ent ways. (Use a computer or calculator to speed up your 
work.) 


Disprove the statement that every positive integer is the 
sum of the cubes of eight nonnegative integers. 


Disprove the statement that every positive integer is the 
sum of at most two squares and a cube of nonnegative 
integers. 


Disprove the statement that every positive integer is the 
sum of 36 fifth powers of nonnegative integers. 


Assuming the truth of the theorem that states that Jn is 
irrational whenever n is a positive integer that is not a 


perfect square, prove that V2 + V3 is irrational. 


. Given the truth values of the propositions p and q in 


fuzzy logic, find the truth value of the disjunction and 
the conjunction of p and q (see Exercises 50 and 51 of 
Section 1.1). 

Given positive integers m and n, interactively play the 
game of Chomp. 

Given a portion of a checkerboard, look for tilings of this 
checkerboard with various types of polyominoes, includ- 
ing dominoes, the two types of triominoes, and larger poly- 
ominoes. 


Use a computational program or programs you have written to do these exercises. 


. Look for positive integers that are not the sum of the cubes 


of nine different positive integers. 


. Look for positive integers greater than 79 that are not the 


sum of the fourth powers of 18 positive integers. 


. Find as many positive integers as you can that can be writ- 


ten as the sum of cubes of positive integers, in two different 
ways, sharing this property with 1729. 


4, 


Try to find winning strategies for the game of Chomp for 
different initial configurations of cookies. 


. Construct the 12 different pentominoes, where a pen- 


tomino is a polyomino consisting of five squares. 


. Find all the rectangles of 60 squares that can be tiled using 


every one of the 12 different pentominoes. 
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Writing Projects 


Respond to these with essays using outside sources. 


. Discuss logical paradoxes, including the paradox of Epi- 


menides the Cretan, Jourdain’s card paradox, and the bar- 
ber paradox, and how they are resolved. 


. Describe how fuzzy logic is being applied to practical 


applications. Consult one or more of the recent books 
on fuzzy logic written for general audiences. 


. Describe some of the practical problems that can be mod- 


eled as satisfiability problems. 


. Explain how satisfiability can be used to model a round 


robin tournament. 


. Describe some of the techniques that have been devised 


to help people solve Sudoku puzzles without the use of a 
computer. 


. Describe the basic rules of WFF’N PROOF, The Game of 


Modern Logic, developed by Layman Allen. Give exam- 
ples of some of the games included in WFF’N PROOF. 


. Read some of the writings of Lewis Carroll on symbolic 


logic. Describe in detail some of the models he used to 
represent logical arguments and the rules of inference he 
used in these arguments. 


10. 


11. 


12. 


13. 


14. 


15. 


. Extend the discussion of Prolog given in Section 1.4, ex- 


plaining in more depth how Prolog employs resolution. 


. Discuss some of the techniques used in computational 


logic, including Skolem’s rule. 

“Automated theorem proving” is the task of using com- 
puters to mechanically prove theorems. Discuss the goals 
and applications of automated theorem proving and the 
progress made in developing automated theorem provers. 
Describe how DNA computing has been used to solve in- 
stances of the satisfiability problem. 

Look up some of the incorrect proofs of famous open 
questions and open questions that were solved since 1970 
and describe the type of error made in each proof. 
Discuss what is known about winning strategies in the 
game of Chomp. 

Describe various aspects of proof strategy discussed by 
George Pólya in his writings on reasoning, including 
[P062], [Po71], and [Po90]. 

Describe a few problems and results about tilings with 
polyominoes, as described in [Go94] and [Ma91], for 
example. 
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Sets 


Basic Structures: Sets, Functions, 
Sequences, Sums, and Matrices 


M= of discrete mathematics is devoted to the study of discrete structures, used to rep- 
resent discrete objects. Many important discrete structures are built using sets, which 
are collections of objects. Among the discrete structures built from sets are combinations, un- 
ordered collections of objects used extensively in counting; relations, sets of ordered pairs that 
represent relationships between objects; graphs, sets of vertices and edges that connect vertices; 
and finite state machines, used to model computing machines. These are some of the topics we 
will study in later chapters. 

The concept of a function is extremely important in discrete mathematics. A function as- 
signs to each element of a first set exactly one element of a second set, where the two sets are 
not necessarily distinct. Functions play important roles throughout discrete mathematics. They 
are used to represent the computational complexity of algorithms, to study the size of sets, to 
count objects, and in a myriad of other ways. Useful structures such as sequences and strings 
are special types of functions. In this chapter, we will introduce the notion of sequences, which 
represent ordered lists of elements. Furthermore, we will introduce some important types of 
sequences and we will show how to define the terms of a sequence using earlier terms. We will 
also address the problem of identifying a sequence from its first few terms. 

In our study of discrete mathematics, we will often add consecutive terms of a sequence of 
numbers. Because adding terms from a sequence, as well as other indexed sets of numbers, is 
such a common occurrence, a special notation has been developed for adding such terms. In this 
chapter, we will introduce the notation used to express summations. We will develop formulae 
for certain types of summations that appear throughout the study of discrete mathematics. For 
instance, we will encounter such summations in the analysis of the number of steps used by an 
algorithm to sort a list of numbers so that its terms are in increasing order. 

The relative sizes of infinite sets can be studied by introducing the notion of the size, or 
cardinality, of a set. We say that a set is countable when it is finite or has the same size as the 
set of positive integers. In this chapter we will establish the surprising result that the set of 
rational numbers is countable, while the set of real numbers is not. We will also show how the 
concepts we discuss can be used to show that there are functions that cannot be computed using 
a computer program in any programming language. 

Matrices are used in discrete mathematics to represent a variety of discrete structures. We 
will review the basic material about matrices and matrix arithmetic needed to represent relations 
and graphs. The matrix arithmetic we study will be used to solve a variety of problems involving 
these structures. 


2.1.1 Introduction 


In this section, we study the fandamental discrete structure on which all other discrete structures 
are built, namely, the set. Sets are used to group objects together. Often, but not always, the 
objects in a set have similar properties. For instance, all the students who are currently enrolled 
in your school make up a set. Likewise, all the students currently taking a course in discrete 
mathematics at any school make up a set. In addition, those students enrolled in your school 
who are taking a course in discrete mathematics form a set that can be obtained by taking the 
elements common to the first two collections. The language of sets is a means to study such 
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Definition 1 


EXAMPLE 1 
EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


Extra > 
Examples 


Beware that mathe- 
maticians disagree 
whether 0 is a natural 
number. We consider it 
quite natural. 


collections in an organized fashion. We now provide a definition of a set. This definition is an 
intuitive definition, which is not part of a formal theory of sets. 


A set is an unordered collection of distinct objects, called elements or members of the set. A 
set is said to contain its elements. We write a € A to denote that a is an element of the set A. 
The notation a é A denotes that a is not an element of the set A. 


It is common for sets to be denoted using uppercase letters. Lowercase letters are usually 
used to denote elements of sets. 

There are several ways to describe a set. One way is to list all the members of a set, when 
this is possible. We use a notation where all members of the set are listed between braces. For 
example, the notation {a, b, c, d} represents the set with the four elements a, b, c, and d. This 
way of describing a set is known as the roster method. 


The set V of all vowels in the English alphabet can be written as V = {a, e, i, 0, u}. < 


The set O of odd positive integers less than 10 can be expressed by O = {1, 3,5, 7, 9}. < 


Although sets are usually used to group together elements with common properties, there is 
nothing that prevents a set from having seemingly unrelated elements. For instance, {a, 2, Fred, 
New Jersey} is the set containing the four elements a, 2, Fred, and New Jersey. < 


Sometimes the roster method is used to describe a set without listing all its members. Some 
members of the set are listed, and then ellipses (...) are used when the general pattern of the 
elements is obvious. 


The set of positive integers less than 100 can be denoted by {1, 2, 3, ..., 99}. < 


Another way to describe a set is to use set builder notation. We characterize all those 
elements in the set by stating the property or properties they must have to be members. The 
general form of this notation is {x | x has property P} and is read “the set of all x such that x has 
property P.” For instance, the set O of all odd positive integers less than 10 can be written as 


O = {x | xis an odd positive integer less than 10}, 
or, specifying the universe as the set of positive integers, as 
O = {x€ Z* |x is odd and x < 10}. 


We often use this type of notation to describe sets when it is impossible to list all the elements 
of the set. For instance, the set Qt of all positive rational numbers can be written as 


Qt={xER|x= p for some positive integers p and q}. 
These sets, each denoted using a boldface letter, play an important role in discrete mathe- 
matics: 


N = {0, 1, 2, 3, ...}, the set of all natural numbers 

Z = {...,—-2,—-1, 0, 1, 2, ...}, the set of all integers 

Zt = {1, 2, 3, ...}, the set of all positive integers 

Q = {p/q |p E Z, q E Z, and q F 0}, the set of all rational numbers 
R, the set of all real numbers 


EXAMPLE 5 


Definition 2 


Links » 
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R+, the set of all positive real numbers 
C, the set of all complex numbers. 


(Note that some people do not consider 0 a natural number, so be careful to check how the term 
natural numbers is used when you read other books.) 

Among the sets studied in calculus and other subjects are intervals, sets of all the real 
numbers between two numbers a and b, with or without a and b. If a and b are real numbers 
with a < b, we denote these intervals by 


[a, b] = {x]a <x <b} 
[a, b) = {xļa <x < b} 
(a,b] = {x|a<x<b} 
(a, b) = {x|a<x <b}. 


Note that [a, b] is called the closed interval from a to b and (a, b) is called the open interval 
from a to b. Each of the intervals [a, b], [a, b), (a, b], and (a, b) contains all the real numbers 
strictly between a and b. The first two of these contain a and the first and third contain b. 


Remark: Some books use the notations [a, b[, Ja, b], and Ja, b[ for [a, b), (a, b], and (a, b), re- 
spectively. 


Sets can have other sets as members, as Example 5 illustrates. 


The set {N, Z, Q, R} is a set containing four elements, each of which is a set. The four elements 
of this set are N, the set of natural numbers; Z, the set of integers; Q, the set of rational numbers; 
and R, the set of real numbers. < 


Remark: Note that the concept of a datatype, or type, in computer science is built upon the con- 
cept of a set. In particular, a datatype or type is the name of a set, together with a set of opera- 
tions that can be performed on objects from that set. For example, boolean is the name of the set 
{0, 1}, together with operators on one or more elements of this set, such as AND, OR, and NOT. 


Because many mathematical statements assert that two differently specified collections of 
objects are really the same set, we need to understand what it means for two sets to be equal. 


Two sets are equal if and only if they have the same elements. Therefore, if A and B are sets, 
then A and B are equal if and only if Vx(x € A + x € B). We write A = B if A and B are equal 
sets. 


GEORG CANTOR (1845-1918) Georg Cantor was born in St. Petersburg, Russia, where his father was a 
successful merchant. Cantor developed his interest in mathematics in his teens. He began his university studies 
in Zurich in 1862, but when his father died he left Zurich. He continued his university studies at the University 
of Berlin in 1863, where he studied under the eminent mathematicians Weierstrass, Kummer, and Kronecker. 
He received his doctor’s degree in 1867, after having written a dissertation on number theory. Cantor assumed 
a position at the University of Halle in 1869, where he continued working until his death. 

Cantor is considered the founder of set theory. His contributions in this area include the discovery that the 
set of real numbers is uncountable. He is also noted for his many important contributions to analysis. Cantor 
also was interested in philosophy and wrote papers relating his theory of sets with metaphysics. 

Cantor married in 1874 and had six children. His melancholy temperament was balanced by his wife’s 
happy disposition. Although he received a large inheritance from his father, he was poorly paid as a professor. 
To mitigate this, he tried to obtain a better-paying position at the University of Berlin. His appointment there was 


blocked by Kronecker, who did not agree with Cantor’s views on set theory. Cantor suffered from mental illness throughout the later 
years of his life. He died in 1918 from a heart attack. 
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{Ø} has one more 
element than Ø. 


Links » 
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EXAMPLE 7 


The sets {1, 3,5} and {3, 5, 1} are equal, because they have the same elements. Note that the 
order in which the elements of a set are listed does not matter. Note also that it does not matter 
if an element of a set is listed more than once, so {1, 3, 3, 3,5, 5,5, 5} is the same as the set 
{1, 3, 5} because they have the same elements. < 


THE EMPTY SET There is a special set that has no elements. This set is called the empty set, 
or null set, and is denoted by Ø. The empty set can also be denoted by { } (that is, we represent 
the empty set with a pair of braces that encloses all the elements in this set). Often, a set of 
elements with certain properties turns out to be the null set. For instance, the set of all positive 
integers that are greater than their squares is the null set. 

A set with one element is called a singleton set. A common error is to confuse the empty 
set Ø with the set {Ø}, which is a singleton set. The single element of the set {Ø} is the empty 
set itself! A useful analogy for remembering this difference is to think of folders in a computer 
file system. The empty set can be thought of as an empty folder and the set consisting of just 
the empty set can be thought of as a folder with exactly one folder inside, namely, the empty 
folder. 


NAIVE SET THEORY Note that the term object has been used in the definition of a set, 
Definition 1, without specifying what an object is. This description of a set as a collection 
of objects, based on the intuitive notion of an object, was first stated in 1895 by the German 
mathematician Georg Cantor. The theory that results from this intuitive definition of a set, and 
the use of the intuitive notion that for any property whatever, there is a set consisting of exactly 
the objects with this property, leads to paradoxes, or logical inconsistencies. This was shown 
by the English philosopher Bertrand Russell in 1902 (see Exercise 50 for a description of one of 
these paradoxes). These logical inconsistencies can be avoided by building set theory beginning 
with axioms. However, we will use Cantor’s original version of set theory, known as naive set 
theory, in this book because all sets considered in this book can be treated consistently using 
Cantor’s original theory. Students will find familiarity with naive set theory helpful if they go on 
to learn about axiomatic set theory. They will also find the development of axiomatic set theory 
much more abstract than the material in this text. We refer the interested reader to [Su72] to 
learn more about axiomatic set theory. 


2.1.2 Venn Diagrams 


Sets can be represented graphically using Venn diagrams, named after the English mathemati- 
cian John Venn, who introduced their use in 1881. In Venn diagrams the universal set U, which 
contains all the objects under consideration, is represented by a rectangle. (Note that the univer- 
sal set varies depending on which objects are of interest.) Inside this rectangle, circles or other 
geometrical figures are used to represent sets. Sometimes points are used to represent the par- 
ticular elements of the set. Venn diagrams are often used to indicate the relationships between 
sets. We show how a Venn diagram can be used in Example 7. 


Draw a Venn diagram that represents V, the set of vowels in the English alphabet. 


Solution: We draw a rectangle to indicate the universal set U, which is the set of the 26 letters 
of the English alphabet. Inside this rectangle we draw a circle to represent V. Inside this circle 
we indicate the elements of V with points (see Figure 1). < 


Definition 3 


EXAMPLE 8 
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FIGURE 1 Venn diagram for the set of vowels. 


2.1.3 Subsets 


It is common to encounter situations where the elements of one set are also the elements of 
a second set. We now introduce some terminology and notation to express such relationships 
between sets. 


The set A is a subset of B, and B is a superset of A, if and only if every element of A is also an 
element of B. We use the notation A C B to indicate that A is a subset of the set B. If, instead, 
we want to stress that B is a superset of A, we use the equivalent notation B D A. (So, A C B 
and B 2 A are equivalent statements.) 


We see that A C B if and only if the quantification 
Va(x EA > xX EB) 


is true. Note that to show that A is not a subset of B we need only find one element x € A with 
x É B. Such an x is a counterexample to the claim that x € A implies x € B. 
We have these useful rules for determining whether one set is a subset of another: 


Showing that A is a Subset of B To show that A C B, show that if x belongs to A then x also 
belongs to B. 


Showing that A is Not a Subset of B To show that A É B, find a single x € A such that x ¢ B. 


The set of all odd positive integers less than 10 is a subset of the set of all positive integers less 
than 10, the set of rational numbers is a subset of the set of real numbers, the set of all computer 


BERTRAND RUSSELL (1872-1970) Bertrand Russell was born into a prominent English family active in 
the progressive movement and having a strong commitment to liberty. He became an orphan at an early age 
and was placed in the care of his father’s parents, who had him educated at home. He entered Trinity College, 
Cambridge, in 1890, where he excelled in mathematics and in moral science. He won a fellowship on the basis 
of his work on the foundations of geometry. In 1910 Trinity College appointed him to a lectureship in logic and 
the philosophy of mathematics. 

Russell fought for progressive causes throughout his life. He held strong pacifist views, and his protests 
against World War I led to dismissal from his position at Trinity College. He was imprisoned for 6 months in 
1918 because of an article he wrote that was branded as seditious. Russell fought for women’s suffrage in Great 
Britain. In 1961, at the age of 89, he was imprisoned for the second time for his protests advocating nuclear 
disarmament. 

Russell’s greatest work was in his development of principles that could be used as a foundation for all 
of mathematics. His most famous work is Principia Mathematica, written with Alfred North Whitehead, 


which attempts to deduce all of mathematics using a set of primitive axioms. He wrote many books on philosophy, physics, and his 
political ideas. Russell won the Nobel Prize for Literature in 1950. 
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science majors at your school is a subset of the set of all students at your school, and the set of 
all people in China is a subset of the set of all people in China (that is, it is a subset of itself). 
Each of these facts follows immediately by noting that an element that belongs to the first set in 
each pair of sets also belongs to the second set in that pair. < 


The set of integers with squares less than 100 is not a subset of the set of nonnegative integers 
because —1 is in the former set [as (—1)? < 100], but not the latter set. The set of people who 
have taken discrete mathematics at your school is not a subset of the set of all computer science 
majors at your school if there is at least one student who has taken discrete mathematics who is 
not a computer science major. < 


Theorem 1 shows that every nonempty set S is guaranteed to have at least two subsets, the 
empty set and the set S itself, that is, Ø C S and S CS. 


For every set S,@)@CS and (@i)SCS. 


Proof: We will prove (i) and leave the proof of (ii) as an exercise. 

Let S be a set. To show that Ø C S, we must show that Vx(x € Ø > x € S) is true. Because 
the empty set contains no elements, it follows that x € @ is always false. It follows that the 
conditional statement x E€ Ø > x € S is always true, because its hypothesis is always false and a 
conditional statement with a false hypothesis is true. Therefore, Vx(x E€ Ø > x € S) is true. This 
completes the proof of (i). Note that this is an example of a vacuous proof. < 


When we wish to emphasize that a set A is a subset of a set B but that A # B, we write A C B 
and say that A is a proper subset of B. For A C Bto be true, it must be the case that A C B and 
there must exist an element x of B that is not an element of A. That is, A is a proper subset of B 
if and only if 


Vxxx EA >x E B)Adx(xEBAx ÉA) 


is true. Venn diagrams can be used to illustrate that a set A is a subset of a set B. We draw the 
universal set U as a rectangle. Within this rectangle we draw a circle for B. Because A is a subset 
of B, we draw the circle for A within the circle for B. This relationship is shown in Figure 2. 

Recall from Definition 2 that sets are equal if they have the same elements. A useful way 
to show that two sets have the same elements is to show that each set is a subset of the other. 
In other words, we can show that if A and B are sets with A C B and B C A, then A = B. That 
is, A = Bif and only if Vx(x € A > x € B) and Yx(x € B > x € A) or equivalently if and only 
if Vx(x € A © x € B), which is what it means for the A and B to be equal. Because this method 
of showing two sets are equal is so useful, we highlight it here. 


JOHN VENN (1834-1923) John Venn was born into a London suburban family noted for its philanthropy. 
He attended London schools and got his mathematics degree from Caius College, Cambridge, in 1857. He was 
elected a fellow of this college and held his fellowship there until his death. He took holy orders in 1859 and, 
after a brief stint of religious work, returned to Cambridge, where he developed programs in the moral sciences. 
Besides his mathematical work, Venn had an interest in history and wrote extensively about his college and 
family. 

Venn’s book Symbolic Logic clarifies ideas originally presented by Boole. In this book, Venn presents a 
systematic development of a method that uses geometric figures, known now as Venn diagrams. Today these 
diagrams are primarily used to analyze logical arguments and to illustrate relationships between sets. In addition 
to his work on symbolic logic, Venn made contributions to probability theory described in his widely used 
textbook on that subject. 
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FIGURE 2. Venn diagram showing that A is a subset of B. 


Showing Two Sets are Equal To show that two sets A and B are equal, show that A C B 
and B CA. 


Sets may have other sets as members. For instance, we have the sets 
A = {@, {a}, {b}, {a, b}} and B = {x | xis a subset of the set {a, b} }. 


Note that these two sets are equal, that is, A = B. Also note that {a} € A, but a ¢ A. 


2.1.4 The Size of a Set 


Sets are used extensively in counting problems, and for such applications we need to discuss 
the sizes of sets. 


Let S be a set. If there are exactly n distinct elements in S where n is a nonnegative integer, 
we say that S is a finite set and that n is the cardinality of S. The cardinality of S is denoted 
by IS]. 


Remark: The term cardinality comes from the common usage of the term cardinal number as 
the size of a finite set. 


Let A be the set of odd positive integers less than 10. Then |A| = 5. < 
Let S be the set of letters in the English alphabet. Then |S] = 26. < 
Because the null set has no elements, it follows that |Ø] = 0. < 


We will also be interested in sets that are not finite. 


A set is said to be infinite if it is not finite. 


The set of positive integers is infinite. < 


We will extend the notion of cardinality to infinite sets in Section 2.5, a challenging topic 
full of surprising results. 
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Definition 7 


2.1.5 Power Sets 
Many problems involve testing all combinations of elements of a set to see if they satisfy some 


property. To consider all such combinations of elements of a set S, we build a new set that has 
as its members all the subsets of S. 


Given a set S, the power set of S is the set of all subsets of the set S. The power set of S is 
denoted by P(S). 


What is the power set of the set {0, 1, 2}? 
Solution: The power set P({0, 1, 2}) is the set of all subsets of {0, 1, 2}. Hence, 
P({0, 1,2) = {Ø, {0}, {1}, {2}, (0, 1}, {0,2}, (1, 2}, {0, 1, 2}}. 


Note that the empty set and the set itself are members of this set of subsets. < 


What is the power set of the empty set? What is the power set of the set {Ø}? 


Solution: The empty set has exactly one subset, namely, itself. Consequently, 
PØ) = {9}. 
The set {Ø} has exactly two subsets, namely, Ø and the set {Ø} itself. Therefore, 


P({B}) = {Ø, {B}}. < 


If a set has n elements, then its power set has 2” elements. We will demonstrate this fact in 
several ways in subsequent sections of the text. 


2.1.6 Cartesian Products 


The order of elements in a collection is often important. Because sets are unordered, a different 
structure is needed to represent ordered collections. This is provided by ordered n-tuples. 


The ordered n-tuple (ay, a, ..., a„) is the ordered collection that has a, as its first element, 
dy as its second element, ..., and a, as its nth element. 


We say that two ordered n-tuples are equal if and only if each corresponding pair of their 
elements is equal. In other words, (a, ad, ...,d,) = (b,, bo, ..., b ) if and only if a; = b,, for 
i= 1,2, ..., n. In particular, ordered 2-tuples are called ordered pairs. The ordered pairs (a, b) 
and (c, d) are equal if and only if a = c and b = d. Note that (a, b) and (b, a) are not equal unless 
a=b. 
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Many of the discrete structures we will study in later chapters are based on the notion of the 
Cartesian product of sets (named after René Descartes). We first define the Cartesian product 
of two sets. 


Let A and B be sets. The Cartesian product of A and B, denoted by A x B, is the set of all 
ordered pairs (a, b), where a € A and b € B. Hence, 


AxB= {(a,b)|aEAAb EB}. 


Let A represent the set of all students at a university, and let B represent the set of all courses 
offered at the university. What is the Cartesian product A x B and how can it be used? 


Solution: The Cartesian product A x B consists of all the ordered pairs of the form (a, b), where a 
is a student at the university and b is a course offered at the university. One way to use the set A X 
B is to represent all possible enrollments of students in courses at the university. Furthermore, 
observe that each subset of A x B represents one possible total enrollment configuration, and 
P(A x B) represents all possible enrollment configurations. < 


What is the Cartesian product of A = {1,2} and B = {a,b,c}? 
Solution: The Cartesian product A x B is 


A xB = {(1, a), (1, b), (1, c), (2, a), (2, b), (2, c)}. < 


Note that the Cartesian products A x B and B x A are not equal unless A = Ø or B = Ø (so 
that A x B = Ø) or A = B (see Exercises 33 and 40). This is illustrated in Example 18. 


Show that the Cartesian product B x A is not equal to the Cartesian product A X B, where A and 
B are as in Example 17. 


RENÉ DESCARTES (1596-1650) René Descartes was born into a noble family near Tours, France, about 
130 miles southwest of Paris. He was the third child of his father’s first wife; she died several days after his 
birth. Because of René’s poor health, his father, a provincial judge, let his son’s formal lessons slide until, at 
the age of 8, René entered the Jesuit college at La Flèche. The rector of the school took a liking to him and 
permitted him to stay in bed until late in the morning because of his frail health. From then on, Descartes spent 
his mornings in bed; he considered these times his most productive hours for thinking. 

Descartes left school in 1612, moving to Paris, where he spent 2 years studying mathematics. He earned 
a law degree in 1616 from the University of Poitiers. At 18 Descartes became disgusted with studying and 
decided to see the world. He moved to Paris and became a successful gambler. However, he grew tired of 
bawdy living and moved to the suburb of Saint-Germain, where he devoted himself to mathematical study. 
When his gambling friends found him, he decided to leave France and undertake a military career. However, he 


never did any fighting. One day, while escaping the cold in an overheated room at a military encampment, he had several feverish 
dreams, which revealed his future career as a mathematician and philosopher. 

After ending his military career, he traveled throughout Europe. He then spent several years in Paris, where he studied mathemat- 
ics and philosophy and constructed optical instruments. Descartes decided to move to Holland, where he spent 20 years wandering 
around the country, accomplishing his most important work. During this time he wrote several books, including the Discours, which 
contains his contributions to analytic geometry, for which he is best known. He also made fundamental contributions to philosophy. 

In 1649 Descartes was invited by Queen Christina to visit her court in Sweden to tutor her in philosophy. Although he was 
reluctant to live in what he called “the land of bears amongst rocks and ice,” he finally accepted the invitation and moved to Sweden. 
Unfortunately, the winter of 1649-1650 was extremely bitter. Descartes caught pneumonia and died in mid-February. 
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Definition 9 


EXAMPLE 19 


EXAMPLE 20 


EXAMPLE 21 


Solution: The Cartesian product B x A is 
B xA = {(a, 1), (a, 2), (b, 1), (b, 2), (c, 1), (c, 2)}. 


This is not equal to A x B, which was found in Example 17. < 


The Cartesian product of more than two sets can also be defined. 


The Cartesian product of the sets A}, A, ...,A,,, denoted by A, X A, X +: XA, is the set of 
ordered n-tuples (a,, do, ..., @,), where a; belongs to A; for i = 1, 2, ...,n. In other words, 


Jae) SS iy X 806 R aa = (Gin Gon coon Gly) || Gn Sela tore = I, coon lin 


What is the Cartesian product A x B x C, where A = {0, 1}, B = {1,2}, and C = {0, 1,2}? 


Solution: The Cartesian product A x B x C consists of all ordered triples (a, b, c), where a € A, 
b € B, and c € C. Hence, 


A x Bx C= {(0, 1, 0), (0, 1, 1), (0, 1, 2), (O, 2, 0), (0, 2, 1), (0, 2, 2), 
(1, 1,0), (1, 1, 1), (1, 1, 2), (1, 2, 0), (1, 2, 1), (1, 2, 2)}. < 


Remark: Note that when A, B, and C are sets, (A x B) x C is not the same as A x B x C (see 
Exercise 41). 


We use the notation A? to denote A x A, the Cartesian product of the set A with itself. 
Similarly, A> =AXAXA,A=AXxXA XA XA, and so on. More generally, 


A" = {(a;, &, ..., a) |a; E A fori = 1,2, ... n}. 


Suppose that A= {1,2}. It follows that A? = {(1, 1), (1, 2), (2, 1), (2,2)} and A? = 
{d, 1, 1), (1, 1, 2), (1, 2, 1), (1, 2, 2), (2, 1, 1), (2, 1, 2), (2, 2, 1), (2, 2, 2)}. < 


A subset R of the Cartesian product A x B is called a relation from the set A to the set B. The 
elements of R are ordered pairs, where the first element belongs to A and the second to B. For 
example, R = {(a, 0), (a, 1), (a, 3), (b, 1), ©, 2), (c, 0), (c, 3)} is a relation from the set {a, b, c} to 
the set {0, 1, 2,3}, and it is also a relation from the set {a, b, c, d, e} to the set {0, 1, 3, 4). (This 
illustrates that a relation need not contain a pair (x, y) for every element x of A.) A relation from 
a set A to itself is called a relation on A. 


What are the ordered pairs in the less than or equal to relation, which contains (a, b) if a < b, 
on the set {0, 1, 2,3}? 


Solution: The ordered pair (a, b) belongs to R if and only if both a and b belong to {0, 1, 2, 3} and 
a < b. Consequently, R = {(0, 0), (0, 1), (0, 2), (0, 3), G, 1), (1, 2), (1, 3), (2, 2), (2, 3), G, 3)}. < 


We will study relations and their properties at length in Chapter 9. 
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2.1.7 Using Set Notation with Quantifiers 


Sometimes we restrict the domain of a quantified statement explicitly by making use of a par- 
ticular notation. For example, Vx € S(P(x)) denotes the universal quantification of P(x) over all 
elements in the set S. In other words, Vx E€ S(P(x)) is shorthand for Yx(x € S > P(x)). Simi- 
larly, 3x € S(P(x)) denotes the existential quantification of P(x) over all elements in S. That is, 
dx € S(P(x)) is shorthand for 3x(x € S$ A P(x)). 


EXAMPLE 22 What do the statements Vx € R (x? > 0) and Ax € Z (x? = 1) mean? 
Solution: The statement Vx € RQ? > 0) states that for every real number x, x2 > 0. This state- 
ment can be expressed as “The square of every real number is nonnegative.” This is a true 
statement. 

The statement 4x € Z(x? = 1) states that there exists an integer x such that x? = 1. This 
statement can be expressed as “There is an integer whose square is 1.” This is also a true state- 
ment because x = 1 is such an integer (as is —1). < 
2.1.8 Truth Sets and Quantifiers 
We will now tie together concepts from set theory and from predicate logic. Given a predicate 
P, and a domain D, we define the truth set of P to be the set of elements x in D for which P(x) 
is true. The truth set of P(x) is denoted by {x € D | P(x)}. 

EXAMPLE 23 What are the truth sets of the predicates P(x), Q(x), and R(x), where the domain is the set of 
integers and P(x) is “|x| = 1,’ Q(x) is “x? = 2,” and R(x) is “|x| = x.” 
Solution: The truth set of P, {x € Z | |x| = 1}, is the set of integers for which |x| = 1. Because 
|x| = 1 when x = 1 or x = —1, and for no other integers x, we see that the truth set of P is the 
set {-1, 1}. 

The truth set of Q, {x € Z| x? = 2}, is the set of integers for which x? = 2. This is the 
empty set because there are no integers x for which x? = 2. 

The truth set of R, {x € Z | |x| = x}, is the set of integers for which |x| = x. Because |x| = x 
if and only if x > 0, it follows that the truth set of R is N, the set of nonnegative integers. <4 

Note that VxP(x) is true over the domain U if and only if the truth set of P is the set U. 
Likewise, 4xP(x) is true over the domain U if and only if the truth set of P is nonempty. 

Exercises 

1. List the members of these sets. 3. Which of the intervals (0, 5), (0, 5], [0, 5), [0,5], (1, 4], 
a) {x |x is areal number such that x? = 1} [2, 3], (2, 3) contains 
b) {x | xis a positive integer less than 12} a). yt 
c) {x | xis the square of an integer and x < 100} c) 2? d) 3? 
d) {x | xis an integer such that x? = 2} e) 4? f) 5? 

2. Use set builder notation to give a description of each of 4. For each of these intervals, list all its elements or explain 
these sets. why it is empty. 
a) {0,3,6,9, 12} a) [a a] b) [a, a) 
b) {-3, —2, —1, 0, 1, 2, 3} c) (a a] d) (a, a) 


c) {m,n, 0, p} 


e) (a, b), where a > b f) [a, b], where a > b 
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5. 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


For each of these pairs of sets, determine whether the first 
is a subset of the second, the second is a subset of the first, 
or neither is a subset of the other. 

a) the set of airline flights from New York to New Delhi, 
the set of nonstop airline flights from New York to 
New Delhi 

b) the set of people who speak English, the set of people 
who speak Chinese 

c) the set of flying squirrels, the set of living creatures 
that can fly 


. For each of these pairs of sets, determine whether the first 


is a subset of the second, the second is a subset of the first, 

or neither is a subset of the other. 

a) the set of people who speak English, the set of people 
who speak English with an Australian accent 

b) the set of fruits, the set of citrus fruits 

c) the set of students studying discrete mathematics, the 
set of students studying data structures 


. Determine whether each of these pairs of sets are equal. 


a) {1, 3, 3,3,5,5,5,5, 5}, {5,3,1} 
b) {{1}}, {4 OH c) Ø, {Ø} 


. Suppose that A = {2, 4, 6}, B = {2,6}, C= {4, 6}, and 


D = {4, 6, 8}. Determine which of these sets are subsets 
of which other of these sets. 


. For each of the following sets, determine whether 2 is an 


element of that set. 

a) {x € R |x is an integer greater than 1} 

b) {x € R |x is the square of an integer} 

c) {2,{2}} d) {{2},{{2}}} 

e) {{2},{2,{2}}} f) {{{2}}} 

For each of the sets in Exercise 9, determine whether {2} 
is an element of that set. 

Determine whether each of these statements is true or 
false. 


a) 0E b) Be {0} 
c) {0} cB d) Øc {0} 
e) {0} € {0} £) {0} c {0} 
g) {8} c {9} 


Determine whether these statements are true or false. 


a) Ø c {Ø} b) DE {Ø, {Ø} } 

c) {9} € {Ø} d) {Ø} € {{9}} 

e) {Ø} c {B, {Ø}} £) {{Ø}} c {9, {Ø}} 

g) {({8}} c {{9}, 18} 

Determine whether each of these statements is true or 
false. 

a) xE {x} b) {x} C {x} c) {x} € {x} 

d efx © OC {x} f) ØE {x} 

Use a Venn diagram to illustrate the subset of odd inte- 
gers in the set of all positive integers not exceeding 10. 
Use a Venn diagram to illustrate the set of all months of 
the year whose names do not contain the letter R in the 
set of all months of the year. 

Use a Venn diagram to illustrate the relationship A C B 
and B CC. 

Use a Venn diagram to illustrate the relationships A C B 
and B C C. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


27. 
28. 
29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


37. 


38. 


Use a Venn diagram to illustrate the relationships A C B 
and A C C. 

Suppose that A, B, and C are sets such that A C B and 
B C C. Show that A C C. 

Find two sets A and B such that A € B and A C B. 

What is the cardinality of each of these sets? 

a) {a} b) {{a}} 

c) {a {a}} d) {a, {a}, {a, {a}}} 
What is the cardinality of each of these sets? 

a) Ø b) {Ø} 

c) {Ø {Ø} d) {Ø, {Ø}, (Ø, {O}}} 

Find the power set of each of these sets, where a and b 
are distinct elements. 

a) {a} b) {a,b} c) {9, {B}} 

Can you conclude that A = B if A and B are two sets with 
the same power set? 

How many elements does each of these sets have where 
a and b are distinct elements? 

a) P({a, b, {a, b}}) 

b) PUG, a, {a}, {{a}}}) 

c) PPW) 

Determine whether each of these sets is the power set of 
a set, where a and b are distinct elements. 

a) Ø b) {9, {a}} 

c) {Ø {a}, {Ø a}} d) {9, {a}, {b}, {a, b}} 
Prove that P(A) C P(B) if and only if A C B. 

Show that if A C C and B C D, then Ax B C CXD 
Let A = {a, b, c, d} and B = {y, z}. Find 

a) AxB. b) BXA. 

What is the Cartesian product A x B, where A is the set 
of courses offered by the mathematics department at a 
university and B is the set of mathematics professors at 
this university? Give an example of how this Cartesian 
product can be used. 

What is the Cartesian product A x B x C, where A is the 
set of all airlines and B and C are both the set of all cities 
in the United States? Give an example of how this Carte- 
sian product can be used. 

Suppose that A x B = Ø, where A and B are sets. What 
can you conclude? 


Let A be a set. Show that Ø x A = A x Ø = Ø. 
Let A = {a, b, c}, B = {x, y}, and C = {0, 1}. Find 


a) AxBxC. b) CxBxA. 

c) CXAXB. d) BxBxB. 
Find A? if 

a) A = {0, 1, 3}. b) A= {1,2,a, b}. 
Find A? if 

a) A = {a}. b) A = {0,a}. 


How many different elements does A x B have if A has m 
elements and B has n elements? 

How many different elements does A x B x C have if A 
has m elements, B has n elements, and C has p elements? 


39. 


40. 


41. 
42. 


43. 


44. 


45. 


46. 


47. 


How many different elements does A” have when A has 
m elements and n is a positive integer? 


Show that A x B # B x A, when A and B are nonempty, 
unless A = B. 


Explain why A x B x C and (A x B) x C are not the same. 


Explain why (A x B) x (C x D) and A x (B x C) x D are 
not the same. 


Prove or disprove that if A and B are sets, then P(A x B) = 
P(A) x P(B). 

Prove or disprove that if A, B, and C are nonempty sets 
and A x B = A x C, then B = C. 

Translate each of these quantifications into English and 
determine its truth value. 

a) YxER Q? 4-1) b) IxxEZ (x* = 2) 

c) YxEZ Q > 0) d) IxER (WC? = x) 
Translate each of these quantifications into English and 
determine its truth value. 

a) IxER (æ = —1) b) IxEZ x+1 >x) 

c) YxEZ (x-1 € Z) d) YxEZ (x* € Z) 

Find the truth set of each of these predicates where the 
domain is the set of integers. 
a) P(x):x7 <3 

c) Rx): 2x+1=0 


b) Qx: x >x 


Set Operations 


48. 


* 49, 


* 50. 


Links » 


* 51. 
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Find the truth set of each of these predicates where the 

domain is the set of integers. 

a) P(x): 8 >1 

ec) RW: x <x 

The defining property of an ordered pair is that two or- 

dered pairs are equal if and only if their first elements are 

equal and their second elements are equal. Surprisingly, 

instead of taking the ordered pair as a primitive con- 

cept, we can construct ordered pairs using basic notions 

from set theory. Show that if we define the ordered pair 

(a, b) to be { {a}, {a, b}}, then (a, b) = (c, d) if and only 

ifa =candb = d. [Hint: First show that { {a}, {a, b}} = 

{{c}, {c, d}} if and only if a = c and b = d.] 

This exercise presents Russell’s paradox. Let S be the 

set that contains a set x if the set x does not belong to 

itself, so that S = {x | x É x}. 

a) Show the assumption that S is a member of S leads to 
a contradiction. 

b) Show the assumption that S is not a member of S leads 
to a contradiction. 

By parts (a) and (b) it follows that the set S cannot be de- 

fined as it was. This paradox can be avoided by restricting 

the types of elements that sets can have. 


b) Q(x): x7 =2 


Describe a procedure for listing all the subsets of a 
finite set. 


2.2.1 Introduction 


Links » 


Definition 1 


EXAMPLE 1 


Two, or more, sets can be combined in many different ways. For instance, starting with the set 
of mathematics majors at your school and the set of computer science majors at your school, we 
can form the set of students who are mathematics majors or computer science majors, the set of 
students who are joint majors in mathematics and computer science, the set of all students not 
majoring in mathematics, and so on. 


Let A and B be sets. The union of the sets A and B, denoted by A U B, is the set that contains 
those elements that are either in A or in B, or in both. 


An element x belongs to the union of the sets A and B if and only if x belongs to A or x belongs 
to B. This tells us that 


AUB={x|xE€AVxEB}. 


The Venn diagram shown in Figure 1 represents the union of two sets A and B. The area that 
represents A U B is the shaded area within either the circle representing A or the circle repre- 
senting B. 

We will give some examples of the union of sets. 


The union of the sets {1,3,5} and {1,2,3} is the set {1,2,3,5}; that is, 
{1, 3,5} vu {1,2,3} = {1, 2,3, 5}. < 
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EXAMPLE 2 


Definition 2 


EXAMPLE 3 


EXAMPLE 4 


Definition 3 


EXAMPLE 5 


Be careful not to 
overcount! 


U U 
A U Bis shaded. A N Bis shaded. 
FIGURE 1 Venn diagram of the FIGURE 2 Venn diagram of the 
union of A and B. intersection of A and B. 


The union of the set of all computer science majors at your school and the set of all mathe- 
matics majors at your school is the set of students at your school who are majoring either in 
mathematics or in computer science (or in both). < 


Let A and B be sets. The intersection of the sets A and B, denoted by A N B, is the set con- 
taining those elements in both A and B. 


An element x belongs to the intersection of the sets A and B if and only if x belongs to A and x 
belongs to B. This tells us that 


ANB={x|xEAAx EB}. 


The Venn diagram shown in Figure 2 represents the intersection of two sets A and B. The shaded 
area that is within both the circles representing the sets A and B is the area that represents the 
intersection of A and B. 

We give some examples of the intersection of sets. 


The intersection of the sets {1,3,5} and {1,2,3} is the set {1,3}; that is, 
{1, 3,5} a {1,2,3} = {1,3}. < 


The intersection of the set of all computer science majors at your school and the set of all mathematics 
majors is the set of all students who are joint majors in mathematics and computer science. < 


Two sets are called disjoint if their intersection is the empty set. 


Let A = {1, 3,5, 7,9} and B = {2, 4, 6, 8, 10}. Because A N B = Ø, A and B are disjoint. < 


We are often interested in finding the cardinality of a union of two finite sets A and B. Note 
that |A| + |B| counts each element that is in A but not in B or in B but not in A exactly once, and 
each element that is in both A and B exactly twice. Thus, if the number of elements that are in 
both A and B is subtracted from |A| + |B|, elements in A N B will be counted only once. Hence, 


JAU B| = |A| + |B] - |A n Bl. 


The generalization of this result to unions of an arbitrary number of sets is called the principle 
of inclusion—exclusion. The principle of inclusion—exclusion is an important technique used in 
enumeration. We will discuss this principle and other counting techniques in detail in Chapters 6 
and 8. 


Definition 4 


EXAMPLE 6 


EXAMPLE 7 


Definition 5 


EXAMPLE 8 


EXAMPLE 9 
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There are other important ways to combine sets. 


Let A and B be sets. The difference of A and B, denoted by A — B, is the set containing those 
elements that are in A but not in B. The difference of A and B is also called the complement 
of B with respect to A. 


Remark: The difference of sets A and B is sometimes denoted by A\B. 


An element x belongs to the difference of A and B if and only if x € A and x ¢ B. This tells us that 
A-B={x|xEGAAx EB}. 

The Venn diagram shown in Figure 3 represents the difference of the sets A and B. The shaded 

area inside the circle that represents A and outside the circle that represents B is the area that 


represents A — B. 
We give some examples of differences of sets. 


The difference of {1, 3,5} and {1, 2, 3} is the set {5}; that is, {1, 3,5} — {1, 2,3} = {5}. This 
is different from the difference of {1, 2,3} and {1, 3, 5}, which is the set {2}. < 


The difference of the set of computer science majors at your school and the set of mathematics 
majors at your school is the set of all computer science majors at your school who are not also 
mathematics majors. < 


Once the universal set U has been specified, the complement of a set can be defined. 


Let U be the universal set. The complement of the set A, denoted by A, is the complement of 
A with respect to U. Therefore, the complement of the set A is U — A. 


Remark: The definition of the complement of A depends on a particular universal set U. This 
definition makes sense for any superset U of A. If we want to identify the universal set U, we 
would write “the complement of A with respect to the set U.” 


An element belongs to A if and only if x ¢ A. This tells us that 


A={xEU|xGA}. 


In Figure 4 the shaded area outside the circle representing A is the area representing A. 
We give some examples of the complement of a set. 


Let A = {a, e, i, o, u} (where the universal set is the set of letters of the English alphabet). Then 
A = {b,c, d, f, g, h, j, k, L m,n, p, q, T, s, t, v, w, X, Y, Z}. <4 


Let A be the set of positive integers greater than 10 (with universal set the set of all positive 
integers). Then A = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}. < 


It is left to the reader (Exercise 21) to show that we can express the difference of A and B 
as the intersection of A and the complement of B. That is, 


A-B=ANB. 
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Set identities and 
propositional 
equivalences are just 
special cases of 
identities for Boolean 
algebra. 


A -B is shaded. 


FIGURE 3 Venn diagram for 
the difference of A and B. 


2.2.2 Set Identities 


Table 1 lists the most important identities of unions, intersections, and complements of sets. 
We will prove several of these identities here, using three different methods. These methods 
are presented to illustrate that there are often many different approaches to the solution of a 
problem. The proofs of the remaining identities will be left as exercises. The reader should note 
the similarity between these set identities and the logical equivalences discussed in Section 1.3. 
(Compare Table 6 of Section 1.6 and Table 1.) In fact, the set identities given can be proved 
directly from the corresponding logical equivalences. Furthermore, both are special cases of 


A is shaded. 


FIGURE 4 Venn diagram for 
the complement of the set A. 


identities that hold for Boolean algebra (discussed in Chapter 12). 


TABLE 1 Set Identities. 


Name 


Identity laws 
AUU=U Domination laws 
ANG=9 


AUA=A Idempotent laws 
ANA=A 
Complementation law 


Commutative laws 


AU(BUC)=(AUB)UC Associative laws 
AN(BNC)=(ANB)NC 
AU(BNC)=(AUB)N(AUC) Distributive laws 
AN(BUC)=(ANB)U(ANC) 


De Morgan’s laws 


AU(ANB)=A Absorption laws 
AN(AUB)= 


Complement laws 


This identity says that 
the complement of the 
intersection of two sets 
is the union of their 
complements. 


EXAMPLE 10 


Extra » 
Examples 


EXAMPLE 11 
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Before we discuss different approaches for proving set identities, we briefly discuss the role 
of Venn diagrams. Although these diagrams can help us understand sets constructed using two 
or three atomic sets (the sets used to construct more complicated combinations of these sets), 
they provide far less insight when four or more atomic sets are involved. Venn diagrams for 
four or more sets are quite complex because it is necessary to use ellipses rather than circles 
to represent the sets. This is necessary to ensure that every possible combination of the sets is 
represented by a nonempty region. Although Venn diagrams can provide an informal proof for 
some identities, such proofs should be formalized using one of the three methods we will now 
describe. 

One way to show that two sets are equal is to show that each is a subset of the other. Recall 
that to show that one set is a subset of a second set, we can show that if an element belongs to 
the first set, then it must also belong to the second set. We generally use a direct proof to do 
this. We illustrate this type of proof by establishing the first of De Morgan’s laws. 


Prove that A NB=AUB. 


Solution: We will prove that the two sets A N B and A U B are equal by showing that each set is 
a subset of the other. Se SS 

First, we will show that A N B C AUB. We do this by showing that if x is in A N B, then it 
must also be in A U B. Now suppose that x € A N B. By the definition of complement, x ¢ A N B. 
Using the definition of intersection, we see that the proposition =((x € A) A (x € B)) is true. 

By applying De Morgan’s law for propositions, we see that =(x € A) or ~(x € B). Using 
the definition of negation of propositions, we have x ¢ A or x ¢ B. Using the definition of the 
complement of a set, we see that this implies that x € A or x € B. Consequently, by the definition 
of union, we see that x € AUB. We have now shown that AN B G AUB. 

Next, we will show that AUBC ANB. We do this by showing that if x is in AU B, then it 
must also be in A N B. Now suppose that x € A U B. By the definition of union, we know that 
x € A orx € B. Using the definition of complement, we see that x ¢ A or x ¢ B. Consequently, 
the proposition (x € A) V ~(x € B) is true. 

By De Morgan’s law for propositions, we conclude that =((x € A) A (x € B)) is true. By the 
definition of intersection, it follows that =(x € A N B). We now use the definition of complement 
to conclude that x € A N B. This shows that A U B CANB. 

Because we have shown that each set is a subset of the other, the two sets are equal, and the 
identity is proved. < 


We can more succinctly express the reasoning used in Example 10 using set builder nota- 
tion, as Example 11 illustrates. 


Use set builder notation and logical equivalences to establish the first De Morgan law A N B = 
AUB. 


Solution: We can prove this identity with the following steps. 


ANB= {x|x ANB} by definition of complement 
= {x | a(x € (AN B))} by definition of does not belong symbol 
= {x|7~@eEAAxeE B)} by definition of intersection 
= {x | a(x E€ A) V a(x € B)} by the first De Morgan law for logical equivalences 
={x|xE€AVXEB} by definition of does not belong symbol 
={x|xE AVxE B} by definition of complement 
={x|xE AU B} by definition of union 


=AUB by meaning of set builder notation 
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EXAMPLE 12 


EXAMPLE 13 


Note that besides the definitions of complement, union, set membership, and set builder 
notation, this proof uses the second De Morgan law for logical equivalences. < 


Proving a set identity involving more than two sets by showing each side of the identity is 
a subset of the other often requires that we keep track of different cases, as illustrated by the 
proof in Example 12 of one of the distributive laws for sets. 


Prove the second distributive law from Table 1, which states that AN (BUC) =(ANB)U 
(AN C) for all sets A, B, and C. 


Solution: We will prove this identity by showing that each side is a subset of the other side. 

Suppose that x € A N (B U C). Then x € A and x € BUC. By the definition of union, it 
follows that x € A, and x € B or x € C (or both). In other words, we know that the compound 
proposition (x € A) A ((x € B) V (x € C)) is true. By the distributive law for conjunction over 
disjunction, it follows that (x € A) A œ € B)) V (x € A) A (x € C)). We conclude that either 
x E€ A and x E B, or x € A and x €E C. By the definition of intersection, it follows that x € ANB 
or x E ANC. Using the definition of union, we conclude that x € (A N B) U (A N C). We con- 
clude tht AN (BUC) C (ANB) U(ANC). 

Now suppose that x € (A N B) U (A N C). Then, by the definition of union, x € A N B or 
x E€ A N C. By the definition of intersection, it follows that x € A and x € B or that x € A and 
x E C. From this we see that x € A, and x € B or x € C. Consequently, by the definition of union 
we see that x € A and x E BUC. Furthermore, by the definition of intersection, it follows that 
x E€ AN (BU C). We conclude that (A N B) U (A N C) CAN(BUC). This completes the proof 
of the identity. < 


Set identities can also be proved using membership tables. We consider each combination 
of the atomic sets (that is, the original sets used to produce the sets on each side) that an element 
can belong to and verify that elements in the same combinations of sets belong to both the sets 
in the identity. To indicate that an element is in a set, a 1 is used; to indicate that an element is 
not in a set, a 0 is used. (The reader should note the similarity between membership tables and 
truth tables.) 


Use a membership table to show that A N (BUC) = (AN B)U(ANC). 


Solution: The membership table for these combinations of sets is shown in Table 2. This table 
has eight rows. Because the columns for AN (BU C) and (AN B) U (A NC) are the same, the 
identity is valid. < 


TABLE 2 A Membership Table for the Distributive Property. 


1 
0 
1 
0 
0 
0 
0 
0 


1 
1 
0 
0 
1 
1 
0 
0 


1 1 
0 1 
1 1 
0 0 
1 1 
0 1 
1 1 
0 0 


O o o oO e e e e 


1 1 
1 1 
1 0 
0 0 
0 0 
0 0 
0 0 
0 0 


O O OOO m me m 


EXAMPLE 14 


EXAMPLE 15 
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Once we have proved set identities, we can use them to prove new identities. In particular, 
we can apply a string of identities, one in each step, to take us from one side of a desired identity 
to the other. It is helpful to explicitly state the identity that is used in each step, as we do in 
Example 14. 


Let A, B, and C be sets. Show that 


AU(BNC) =(CUB)NA. 


Solution: We have 


AU(BNC) =An (BNC) bythe first De Morgan law 
=An (B U C) by the second De Morgan law 
= (B U ©) nA by the commutative law for intersections 


= (C UB yn A by the commutative law for unions. < 


We summarize the three different ways for proving set identities in Table 3. 


TABLE 3 Methods of Proving Set Identities. 


Subset method Show that each side of the identity is a subset of the other side. 


Membership table For each possible combination of the atomic sets, show that an element 


in exactly these atomic sets must either belong to both sides or belong to 


neither side 
Apply existing identities Start with one side, transform it into the other side using a sequence of 
steps by applying an established identity. 


2.2.3 Generalized Unions and Intersections 


Because unions and intersections of sets satisfy associative laws, the sets AUBUC and 
ANBNC are well defined; that is, the meaning of this notation is unambiguous when A, B, 
and C are sets. That is, we do not have to use parentheses to indicate which operation comes 
first because A U (BuU C) = (AUB)UC and AN(BNC) = (ANB)NC. Note that AUBUC 
contains those elements that are in at least one of the sets A, B, and C, and that A N BN C con- 
tains those elements that are in all of A, B, and C. These combinations of the three sets, A, B, 
and C, are shown in Figure 5. 


Let A = {0, 2,4,6,8}, B= {0, 1,2,3,4}, and C= {0,3,6,9}. What are AUBUC and 
ANBnC? 


Solution: The set AU B uU C contains those elements in at least one of A, B, and C. Hence, 
AUBUC= {0, 1, 2, 3, 4, 6, 8, 9}. 
The set A N B N C contains those elements in all three of A, B, and C. Thus, 


ANBnNC= {0}. < 
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(a) A U B U Cis shaded. (b) A AB NCis shaded. 


FIGURE 5 The union and intersection of A, B, and C. 


We can also consider unions and intersections of an arbitrary number of sets. We introduce 
these definitions. 


Definition 6 The union of a collection of sets is the set that contains those elements that are members of 
at least one set in the collection. 


We use the notation 


to denote the union of the sets A,, A>, ..., A 


n° 


Definition 7 The intersection of a collection of sets is the set that contains those elements that are members 
of all the sets in the collection. 


We use the notation 


to denote the intersection of the sets A,, A», ..., A„. We illustrate generalized unions and inter- 
sections with Example 16. 


EXAMPLE 16 Fori=1,2,...,letA; = {41+ Li+2,... }. Then, 


Extra > 
Examples 


ee erriren 
i=1 


i=] 


and 


()Ar=() Git Li+2,..} = {nnn N} =A, 
iel i=1 


EXAMPLE 17 


EXAMPLE 18 
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We can extend the notation we have introduced for unions and intersections to other families of 
sets. In particular, to denote the union of the infinite family of sets A4, A, ...,A,, ..., we use the 
notation 


I 


A UAU- UA 


Similarly, the intersection of these sets is denoted by 


A NANA NA, A = (YA: 
i=1 


More generally, when J is a set, the notations (},-,A; and |J;er4; are used to denote 
the intersection and union of the sets A; for i € J, respectively. Note that we have {),-,A; = 
{x | Vi € I (x € A;)} and U,-, A; = {x | Ii E T(x EA} 


iel 


Suppose that A; = {1, 2, 3,..., i} fori = 1, 2,3, ... . Then, 


ve UJ .23,.08 = {1,2,3,..}=Zt 


(ee (1.2.3, 8 = {1}. 


To see that the union of these sets is the set of positive integers, note that every positive 
integer n is in at least one of the sets, because it belongs to A, = {1, 2, ..., n}, and every element 
of the sets in the union is a positive integer. To see that the intersection of these sets is the set 
{1}, note that the only element that belongs to all the sets A,, A,, ... is 1. To see this note that 
A, = {1} and 1 € A; fori=1,2,.... < 


2.2.4 Computer Representation of Sets 


There are various ways to represent sets using a computer. One method is to store the elements 
of the set in an unordered fashion. However, if this is done, the operations of computing the 
union, intersection, or difference of two sets would be time consuming, because each of these 
operations would require a large amount of searching for elements. We will present a method for 
storing elements using an arbitrary ordering of the elements of the universal set. This method 
of representing sets makes computing combinations of sets easy. 

Assume that the universal set U is finite (and of reasonable size so that the number of 
elements of U is not larger than the memory size of the computer being used). First, specify an 
arbitrary ordering of the elements of U, for instance a}, ay, ...,a,. Represent a subset A of U 
with the bit string of length n, where the ith bit in this string is 1 if a; belongs to A and is 0 if a; 
does not belong to A. Example 18 illustrates this technique. 


Let U = {1, 2,3, 4,5, 6, 7, 8, 9, 10}, and the ordering of elements of U has the elements in in- 
creasing order; that is, a; = i. What bit strings represent the subset of all odd integers in U, the 
subset of all even integers in U, and the subset of integers not exceeding 5 in U? 
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EXAMPLE 19 


EXAMPLE 20 


Solution: The bit string that represents the set of odd integers in U, namely, {1, 3, 5, 7,9}, has a 
one bit in the first, third, fifth, seventh, and ninth positions, and a zero elsewhere. It is 


10 1010 1010. 


(We have split this bit string of length ten into blocks of length four for easy reading.) Similarly, 
we represent the subset of all even integers in U, namely, {2, 4, 6, 8, 10}, by the string 


01 0101 0101. 


The set of all integers in U that do not exceed 5, namely, {1, 2, 3, 4,5}, is represented by the 
string 


11 1110 0000. ‘ 


Using bit strings to represent sets, it is easy to find complements of sets and unions, inter- 
sections, and differences of sets. To find the bit string for the complement of a set from the bit 
string for that set, we simply change each 1 to a 0 and each 0 to 1, because x € A if and only if 


x ¢ A. Note that this operation corresponds to taking the negation of each bit when we associate 
a bit with a truth value—with 1 representing true and 0 representing false. 


We have seen that the bit string for the set {1,3,5,7,9} (with universal set {1, 2, 3, 4, 
5, 6, 7, 8, 9, 10}) is 


10 1010 1010. 


What is the bit string for the complement of this set? 


Solution: The bit string for the complement of this set is obtained by replacing Os with 1s and 
vice versa. This yields the string 


01 0101 0101, 
which corresponds to the set {2, 4, 6, 8, 10}. < 


To obtain the bit string for the union and intersection of two sets we perform bitwise Boolean 
operations on the bit strings representing the two sets. The bit in the ith position of the bit string 
of the union is 1 if either of the bits in the ith position in the two strings is 1 (or both are 1), and 
is O when both bits are 0. Hence, the bit string for the union is the bitwise OR of the bit strings 
for the two sets. The bit in the ith position of the bit string of the intersection is 1 when the 
bits in the corresponding position in the two strings are both 1, and is 0 when either of the two 
bits is O (or both are). Hence, the bit string for the intersection is the bitwise AND of the bit 
strings for the two sets. 


The bit strings for the sets {1, 2, 3, 4,5} and {1, 3,5, 7,9} are 11 1110 0000 and 10 1010 1010, 
respectively. Use bit strings to find the union and intersection of these sets. 


Solution: The bit string for the union of these sets is 
11 1110 0000 v 10 1010 1010 = 11 1110 1010, 


which corresponds to the set {1, 2,3, 4,5, 7,9}. The bit string for the intersection of these 
sets is 


11 1110 0000 A 10 1010 1010 = 10 1010 0000, 


which corresponds to the set {1, 3, 5}. < 


EXAMPLE 21 


©Dinodia Photos/Alamy 
Stock Photo 
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2.2.5 Multisets 


Sometimes the number of times that an element occurs in an unordered collection matters. A 
multiset (short for multiple-membership set) is an unordered collection of elements where an 
element can occur as a member more than once. We can use the same notation for a multiset 
as we do for a set, but each element is listed the number of times it occurs. (Recall that in a set, 
an element either belongs to a set or it does not. Listing it more than once does not affect the 
membership of this element in the set.) So, the multiset denoted by {a, a, a, b, b} is the multiset 
that contains the element a thrice and the element b twice. When we use this notation, it must 
be clear that we are working with multisets and not ordinary sets. We can avoid this ambiguity 


by using an alternate notation for multisets. The notation {7 : 44, my ` dy, ..., m,-a,} denotes 
the multiset with element a, occurring m, times, element a, occurring m, times, and so on. 
The numbers m,, i = 1, 2, ..., r, are called the multiplicities of the elements a,, i = 1, 2, ..., r. 


(Elements not in a multiset are assigned O as their multiplicity in this set.) The cardinality of 
a multiset is defined to be the sum of the multiplicities of its elements. The word multiset was 
introduced by Nicolaas Govert de Bruijn in the 1970s, but the concept dates back to the 12th 
century work of the Indian mathematician Bhaskaracharya. 

Let P and Q be multisets. The union of the multisets P and Q is the multiset in which the 
multiplicity of an element is the maximum of its multiplicities in P and Q. The intersection of P 
and Q is the multiset in which the multiplicity of an element is the minimum of its multiplicities 
in P and Q. The difference of P and Q is the multiset in which the multiplicity of an element is 
the multiplicity of the element in P less its multiplicity in Q unless this difference is negative, 
in which case the multiplicity is 0. The sum of P and Q is the multiset in which the multiplic- 
ity of an element is the sum of multiplicities in P and Q. The union, intersection, and differ- 
ence of P and Q are denoted by P U Q, P N Q, and P — Q, respectively (where these operations 
should not be confused with the analogous operations for sets). The sum of P and Q is denoted 
by P+@Q. 


Suppose that P and Q are the multisets {4-a,1-b,3-c} and {3-a,4-b,2-d}, respectively. 
Find P U Q, PN Q, P — Q, and P + Q. 


Solution: We have 


P U Q = {max(4, 3) - a, max(1, 4) - b, max(3, 0) - c, max(0, 2) - d} 
= {4-4,4-b,3-c,2-d}, 


P NQ = {min(4, 3) - a, min(1, 4) - b, min(3, 0) - c, min(0, 2) - d} 
= {3-a,1-b,0-c,0-d} = {3-a1-D}, 


BHASKARACHARYA (1114-1185) Bhaskaracharya was born in Bijapur in the Indian state of Karnataka. 
(Bhaskaracharya’s name was actually Bhaskara, but the title Acharya, which means teacher, was added honorif- 
ically.) His father was a well-known scholar and a famous astrologer. Bhaskaracharya was head of the astronom- 
ical observatory at Ujjain, the leading Indian mathematical center of the day. He is considered to be the greatest 
mathematician of medieval India. Bhaskaracharya made discoveries in many parts of mathematics, including 
geometry, plane and spherical trigonometry, algebra, number theory, and combinatorics. Bhaskaracharya de- 
scribed the principles of differential calculus, which he applied to astronomical problems, predating the works 
of Newton and Leibniz by more than 500 years. In number theory he made many discoveries about Diophantine 
equations, the study of the solution in integers of equations, which were rediscovered more than 600 years later. 
His greatest work is the Crown of Treatises (Siddhanta Shiromani), which includes four main parts, covering 
arithmetic, algebra, mathematics of the planets, and spheres. 
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P — Q = {max(4 — 3, 0) - a, max(1 — 4, 0) - b, max(3 — 0, 0) - c, max(O — 2, 0) - d} 
= {1-4,0-b,3-c,0-d} = {1-a,3-c}, and 


P+Q={4+3)-4(1+4)-b,8+4+0)-c,(0+2)-d} 


= {7-a,5-b,3-c,2-d}. 


Exercises 


1. 


3. 


Let A be the set of students who live within one mile of 
school and let B be the set of students who walk to classes. 
Describe the students in each of these sets. 

a) ANB b) AUB 

c) A-B d) B-A 


. Suppose that A is the set of sophomores at your school 


and B is the set of students in discrete mathematics at 

your school. Express each of these sets in terms of A 

and B. 

a) the set of sophomores taking discrete mathematics in 
your school 

b) the set of sophomores at your school who are not tak- 
ing discrete mathematics 

c) the set of students at your school who either are 
sophomores or are taking discrete mathematics 

d) the set of students at your school who either are not 
sophomores or are not taking discrete mathematics 


Let A = {1, 2,3, 4,5} and B = {0, 3, 6}. Find 


a) AUB. b) ANB. 
c) A-B. d) B-A. 
. Let A = {a,b,c,d,e} and B = {a, b, c, d, e, f, g, h}. Find 
a) AUB. b) ANB. 
c) A-B. d) B-A. 


In Exercises 5-10 assume that A is a subset of some underly- 
ing universal set U. 


5. 


12. 


Prove the complementation law in Table 1 by showing 


that A = A. 


. Prove the identity laws in Table 1 by showing that 


a) AUQ=A. b) ANU =A. 

. Prove the domination laws in Table 1 by showing that 
a) AUU =V. b) AnG=9. 

. Prove the idempotent laws in Table 1 by showing that 
a) AUA =A. b) ANA =A. 

. Prove the complement laws in Table 1 by showing that 
a) AUA=U. b) ANA=9. 

. Show that 

a) A- =A. b) Ø-A =ð. 


. Let A and B be sets. Prove the commutative laws from 


Table 1 by showing that 

a) AUB=BUA. 

b) ANB=BnNA. 

Prove the first absorption law from Table 1 by showing 
that if A and B are sets, then A U (A N B) = A. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


Prove the second absorption law from Table 1 by show- 
ing that if A and B are sets, then A N (A U B) = A. 

Find the sets A and B if A — B = {1,5,7,8}, B-A = 
{2, 10}, and A N B = {3, 6, 9}. 

Prove the second De Morgan law in Table 1 by showing 
that if A and B are sets, then A U B = ANB 

a) by showing each side is a subset of the other side. 

b) using a membership table. 

Let A and B be sets. Show that 

a) (ANB) CA. b) AC (AUB). 

c) A-BCA. d) An(B-A)=9. 

e) AU(B—A) =AUB. 

Show that if A and B are sets in a universe U then A C B 
if and only if AUB =U. 

Given sets A and B in a universe U, draw the Venn dia- 
grams of each of these sets. 

a) A> B={xEU|xEA>xeEB} 

b) AGB={xEU|xEAvcxeEB} 

Show that if A, B, and C are sets, then AN BN C = AU 
BUC 

a) by showing each side is a subset of the other side. 

b) using a membership table. 

Let A, B, and C be sets. Show that 

a) (AUB) C(AUBUC). 

b) (ANBNC) C(ANB). 

c) (A-B)-CCA-C. 

d) A-OQ)n(C-B)=9. 

e) (B—A)U(C—-A) =(BUC)-A. 

Show that if A and B are sets, then 

a) A~B=AnB. 

b) (ANB) U(ANB) =A. 

Show that if A and B are sets with A C B, then 

a) AUB=B. 

b) ANB=A. 

Prove the first associative law from Table 1 by show- 
ing that if A, B, and C are sets, then AU(BUC) = 
(AUB)UC. 

Prove the second associative law from Table 1 by show- 
ing that if A, B, and C are sets, then AN(BNC) = 
(ANB)NC. 

Prove the first distributive law from Table 1 by showing 
that if A, B, and C are sets, then A U (BNC) = (AUB)N 
(AUC). 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


37. 


Let A, B, and C be sets. Show that (A — B)- C = 
(A-C)- (B-C). 


Let A = {0, 2,4,6, 8, 10}, B= {0, 1,2,3,4,5,6}, and 
C = {4,5, 6, 7, 8, 9, 10}. Find 

a) ANBNC. b) AUBUC. 

c) (AUB)NC. d) (ANB)UC. 

Draw the Venn diagrams for each of these combinations 


of the sets A, B, and C. 

a) AN(BUC) b) ANBNC 

c) (A-B)U(A-OC)UB-C) 

Draw the Venn diagrams for each of these combinations 
of the sets A, B, and C. 

a) AN(B-C) 

© (ANB) U(ANC) 
Draw the Venn diagrams for each of these combinations 
of the sets A, B, C, and D. 


a) (ANB)U(CND) b) 
c A—-(BNCND) 


b) (ANB)UANC) 


UBUCUD 


What can you say about the sets A and B if we know that 
a) AUB=A? b) ANB=A? 
c) A-B=A? d) ANB=BnA? 


e A-B=B-A? 

Can you conclude that A = B if A, B, and C are sets such 
that 

a) AUC=BUuUC? b) ANC=BnC? 

c) AUC=BUCandANC=BnNC? 

Let A and B be subsets of a universal set U. Show that 
A C Bif and only if B CA. 

Let A, B, and C be sets. Use the the identity A — B= 
A N B, which holds for any sets A and B, and the identities 
from Table 1 to show that (A — B)N(B-—C)n(A-C) 
=Ø. 

Let A, B, and C be sets. Use the identities in Table 1 to 
show that (AUBIN BUON (AUC) =ANBNC. 
Prove or disprove that for all sets A, B, and C, we have 

a) AX(BUQC)=(AXB)U(AXC). 

b) Ax(BNQC)=(AXB)N(AXC). 

Prove or disprove that for all sets A, B, and C, we have 

a) Ax(B-C)=(AxB)-AxC). 

b) AX(BUC)=AXx(BUC). 


The symmetric difference of A and B, denoted by A @ B, is 
the set containing those elements in either A or B, but not in 
both A and B. 


38. 
39. 


40. 


41. 
42. 
43. 


Find the symmetric difference of {1, 3,5} and {1, 2, 3}. 
Find the symmetric difference of the set of computer sci- 
ence majors at a school and the set of mathematics majors 
at this school. 


Draw a Venn diagram for the symmetric difference of the 
sets A and B. 


Show that A @ B = (AUB) — (A N B). 


Show that A @ B = (A — B) u (B — A). 

Show that if A is a subset of a universal set U, then 
a) A@A=Z. b) A@H=A. 

ce) AQU =A. d) A@A=U. 


44. 


45. 
* 46. 


* 47. 


48. 


49. 


50. 


51. 


* 52. 


53. 


54. 


55. 


56. 


57. 


58. 
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Show that if A and B are sets, then 

a) APBB=BOA. b) A®B)@B=A. 
What can you say about the sets A and B if A @ B =A? 
Determine whether the symmetric difference is associa- 
tive; that is, if A, B, and C are sets, does it follow that 
A®BBOC)=AGB)OC? 

Suppose that A, B, and C are sets such that A®@C= 
B @ C. Must it be the case that A = B? 

If A, B, C, and D are sets, does it follow that (A @ B) ® 
(C ® D) = (A @ C) ® (B @ D)? 

If A, B, C, and D are sets, does it follow that (A @ B) ® 
(C®D)=AGD)GBSC)? 

Show that if A and B are finite sets, then A U B is a finite 
set. 

Show that if A is an infinite set, then whenever B is a set, 
A U B is also an infinite set. 

Show that if A, B, and C are sets, then 


|JAUBUC|= |A| + |B] +|C]-|AnB| 
-—|ANC|-|BNC|+|ANBnCl. 


(This is a special case of the inclusion—exclusion princi- 
ple, which will be studied in Chapter 8.) 
Let A; = {1, 2, 3, ..., i} fori = 1, 2,3,.... Find 


a) Un b) ya 
i=1 i=1 


Let A; = {..., -2,-1,0, 1,..., i}. Find 


a) Ua. b) Na. 
i=1 i=1 


Let A, be the set of all nonempty bit strings (that is, bit 
strings of length at least one) of length not exceeding i. 


Find 

a) JA. b) (Ai. 
i=l i=l 

Find [JZ A; and ();°, A; if for every positive integer i, 

a) A; = {iit1,i+2,...}. 

b) A, = {0, i}. 

c) A; = (0, i), that is, the set of real numbers x with 
O0<x<i. 

d) A; = (i, œ), that is, the set of real numbers x with 
x>i. 

Find J, A; and ();°, A; if for every positive integer i, 


a) A; = {—i,—i + 1, ...,—1,0, 1, ...,i— 1, i}. 

b) A; = {-i i}. 

c) A; =[-i, i], that is, the set of real numbers x with 
-i<x<i. 


d) A; = [i co), that is, the set of real numbers x with 
x>i. 

Suppose that the universal set is U = {1,2,23,4, 

5, 6, 7, 8,9, 10}. Express each of these sets with bit 

strings where the ith bit in the string is 1 if i is in the 

set and 0 otherwise. 

a) {3,4,5} 

b) {1, 3, 6, 10} 

c) {2, 3,4, 7, 8,9} 
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59. Using the same universal set as in the last exercise, find 
the set specified by each of these bit strings. 
a) 11 1100 1111 
b) 01 0111 1000 
c) 100000 0001 

60. What subsets of a finite universal set do these bit strings 
represent? 

a) the string with all zeros 
b) the string with all ones 

61. What is the bit string corresponding to the difference of 
two sets? 

62. What is the bit string corresponding to the symmetric dif- 
ference of two sets? 

63. Show how bitwise operations on bit strings can be 
used to find these combinations of A = {a,b,c,d,e}, 
B={bhodgptv}, C={ceio0,u,x yz}, and 
D = {d,e,h,i, n, o, t, u, x, y}. 

a) AUB b) ANB 
c) (AUD)N(BUC) d) AUBUCUD 

64. How can the union and intersection of n sets that all are 
subsets of the universal set U be found using bit strings? 

The successor of the set A is the set A U {A}. 

65. Find the successors of the following sets. 

a) {1,2,3} b) Ø 
c) {Ø} d) {9, {Ø}} 

66. How many elements does the successor of a set with n 
elements have? 

67. Let A and B be the multisets {3-a,2-b,1-c} and 
{2 -a,3 - b, 4 - d}, respectively. Find 
a) AUB. b) ANB. 

d) B-A. e) A+B. 

68. Assume thata € A, where A is a set. Which of these state- 
ments are true and which are false, where all sets shown 
are ordinary sets, and not multisets. Explain each answer. 


c) A-B. 


a) {a,a} U {a,a,a} = {a,a,a, a, a} 
b) {a,a} U {a,a, a} = {a} 
c) {a, a} N {a,a,a} = {a, a} 
d) {a,a} N {a,a, a} = {a} 
e) {a,a,a} — {a,a} = {a} 
69. Answer the same questions as posed in Exercise 68 where 
all sets are multisets, and not ordinary sets. 
70. Suppose that A is the multiset that has as its elements 
the types of computer equipment needed by one depart- 
ment of a university and the multiplicities are the num- 
ber of pieces of each type needed, and B is the analogous 
multiset for a second department of the university. For 
instance, A could be the multiset {107 - personal comput- 
ers, 44 - routers, 6 - servers} and B could be the multiset 
{14 - personal computers, 6 - routers, 2 - mainframes}. 
a) What combination of A and B represents the equip- 
ment the university should buy assuming both depart- 
ments use the same equipment? 

b) What combination of A and B represents the equip- 
ment that will be used by both departments if both 
departments use the same equipment? 


c) What combination of A and B represents the equip- 
ment that the second department uses, but the first de- 
partment does not, if both departments use the same 
equipment? 

d) What combination of A and B represents the equip- 
ment that the university should purchase if the depart- 
ments do not share equipment? 


The Jaccard similarity J(A, B) of the finite sets A and 
B is J(A, B) = |A A B|/|A U B|, with J(Ø, Ø) = 1. The Jac- 
card distance d,(A, B) between A and B equals d,(A, B) = 
1 — J(A, B). 
71. Find J(A, B) and d,(A, B) for these pairs of sets. 
a) A= {1,3,5}, B= {2,4, 6} 
b) A= {1,2,3,4}, B= {3, 4,5,6} 
c) A= {1,2,3,4,5, 6}, B= {1, 2,3, 4, 5,6} 
d) A= {1}, B= {1,2,3,4,5, 6} 
72. Prove that each of the properties in parts (a)—(d) holds 
whenever A and B are finite sets. 
a) J(A, A) = 1 andd,(A, A) = 0 
b) J(A, B) = J(B, A) and d,(A, B) = d,(B, A) 
c) J(A, B) = 1 andd,(A, B) = 0 if and only if A = B 
d) 0 < J(A, B) < 1 and 0 <d,(A, B) < 1 
**e) Show that if A, B, and C are sets, then d,(A, C) < 
d,(A, B) + d,(B, C). (This inequality is known as the 
triangle inequality and together with parts (a), (b), 
and (c) implies that d; is a metric.) 


Fuzzy sets are used in artificial intelligence. Each element 
in the universal set U has a degree of membership, which 
is a real number between 0 and 1 (including 0 and 1), in a 
fuzzy set S. The fuzzy set S is denoted by listing the elements 


Links with their degrees of membership (elements with 0 degree of 


membership are not listed). For instance, we write {0.6 Alice, 

0.9 Brian, 0.4 Fred, 0.1 Oscar, 0.5 Rita} for the set F (of fa- 

mous people) to indicate that Alice has a 0.6 degree of mem- 

bership in F, Brian has a 0.9 degree of membership in F, Fred 
has a 0.4 degree of membership in F, Oscar has a 0.1 degree 
of membership in F, and Rita has a 0.5 degree of membership 
in F (so that Brian is the most famous and Oscar is the least 
famous of these people). Also suppose that R is the set of rich 
people with R = {0.4 Alice, 0.8 Brian, 0.2 Fred, 0.9 Oscar, 

0.7 Rita}. 

73. The complement of a fuzzy set S is the set S, with the 
degree of the membership of an element in § equal to 
1 minus the degree of membership of this element in S. 
Find F (the fuzzy set of people who are not famous) and 
R (the fuzzy set of people who are not rich). 

74. The union of two fuzzy sets S and T is the fuzzy set S U T, 
where the degree of membership of an element in SUT 
is the maximum of the degrees of membership of this el- 
ement in S and in T. Find the fuzzy set F U R of rich or 
famous people. 

75. The intersection of two fuzzy sets S and T is the fuzzy 
set S N T, where the degree of membership of an element 
in SMT is the minimum of the degrees of membership 
of this element in S and in T. Find the fuzzy set F N R of 
rich and famous people. 
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Functions 


Definition 1 


Assessment $ 


Definition 2 


2.3.1 Introduction 


In many instances we assign to each element of a set a particular element of a second set (which 
may be the same as the first). For example, suppose that each student in a discrete mathematics 
class is assigned a letter grade from the set {A, B, C, D, F}. And suppose that the grades are A 
for Adams, C for Chou, B for Goodfriend, A for Rodriguez, and F for Stevens. This assignment 
of grades is illustrated in Figure 1. 

This assignment is an example of a function. The concept of a function is extremely im- 
portant in mathematics and computer science. For example, in discrete mathematics functions 
are used in the definition of such discrete structures as sequences and strings. Functions are 
also used to represent how long it takes a computer to solve problems of a given size. Many 
computer programs and subroutines are designed to calculate values of functions. Recursive 
functions, which are functions defined in terms of themselves, are used throughout computer 
science; they will be studied in Chapter 5. This section reviews the basic concepts involving 
functions needed in discrete mathematics. 


Let A and B be nonempty sets. A function f from A to B is an assignment of exactly one 
element of B to each element of A. We write f(a) = b if b is the unique element of B assigned 
by the function f to the element a of A. If f is a function from A to B, we write f : A > B. 


Remark: Functions are sometimes also called mappings or transformations. 


Functions are specified in many different ways. Sometimes we explicitly state the assign- 
ments, as in Figure 1. Often we give a formula, such as f(x) = x + 1, to define a function. Other 
times we use a computer program to specify a function. 

A function f : A > B can also be defined in terms of a relation from A to B. Recall from 
Section 2.1 that a relation from A to B is just a subset of A x B. A relation from A to B that 
contains one, and only one, ordered pair (a, b) for every element a € A, defines a function f 
from A to B. This function is defined by the assignment f(a) = b, where (a, b) is the unique 
ordered pair in the relation that has a as its first element. 


If f is a function from A to B, we say that A is the domain of f and B is the codomain of f. 
If f(a) = b, we say that b is the image of a and a is a preimage of b. The range, or image, of 
f is the set of all images of elements of A. Also, if f is a function from A to B, we say that f 
maps A to B. 


Adams e eA 
Chou e eB 
Goodfriend e ec 
Rodriguez @ eD 
Stevens e =e F 
FIGURE 1 Assignment of grades in a discrete mathematics class. 


148 2/ Basic Structures: Sets, Functions, Sequences, Sums, and Matrices 


EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


Extra 
Examples 


7 


FIGURE 2 The function f maps A to B. 


Figure 2 represents a function f from A to B. 


Remark: Note that the codomain of a function from A to B is the set of all possible values of 
such a function (that is, all elements of B), and the range is the set of all values of f(a) for a € A, 
and is always a subset of the codomain. That is, the codomain is the set of possible values of the 
function and the range is the set of all elements of the codomain that are achieved as the value 
of f for at least one element of the domain. 


When we define a function we specify its domain, its codomain, and the mapping of el- 
ements of the domain to elements in the codomain. Two functions are equal when they have 
the same domain, have the same codomain, and map each element of their common domain 
to the same element in their common codomain. Note that if we change either the domain or 
the codomain of a function, then we obtain a different function. If we change the mapping of 
elements, then we also obtain a different function. 

Examples 1-5 provide examples of functions. In each case, we describe the domain, the 
codomain, the range, and the assignment of values to elements of the domain. 


What are the domain, codomain, and range of the function that assigns grades to students de- 
scribed in the first paragraph of the introduction of this section? 


Solution: Let G be the function that assigns a grade to a student in our discrete mathematics class. 
Note that G(Adams) = A, for instance. The domain of G is the set {Adams, Chou, Goodfriend, 
Rodriguez, Stevens}, and the codomain is the set {A, B, C, D, F}. The range of G is the set 
{A, B, C, F}, because each grade except D is assigned to some student. < 


Let R be the relation with ordered pairs (Abdul, 22), (Brenda, 24), (Carla, 21), (Desire, 22), 
(Eddie, 24), and (Felicia, 22). Here each pair consists of a graduate student and this student’s 
age. Specify a function determined by this relation. 


Solution: If f is a function specified by R, then f(Abdul) = 22, (Brenda) = 24, 
f (Carla) = 21, f(Desire) = 22, f(Eddie) = 24, and f(Felicia) = 22. [Here, f(x) is the age of x, 
where x is a student.] For the domain, we take the set {Abdul, Brenda, Carla, Desire, Eddie, 
Felicia}. We also need to specify a codomain, which needs to contain all possible ages of stu- 
dents. Because it is highly likely that all students are less than 100 years old, we can take the 
set of positive integers less than 100 as the codomain. (Note that we could choose a different 
codomain, such as the set of all positive integers or the set of positive integers between 10 and 
90, but that would change the function. Using this codomain will also allow us to extend the 
function by adding the names and ages of more students later.) The range of the function we 
have specified is the set of different ages of these students, which is the set {21, 22, 24}. < 


Let f be the function that assigns the last two bits of a bit string of length 2 or greater to that 
string. For example, f(11010) = 10. Then, the domain of f is the set of all bit strings of length 
2 or greater, and both the codomain and range are the set {00, 01, 10, 11}. < 


EXAMPLE 4 


EXAMPLE 5 
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Let f: Z > Z assign the square of an integer to this integer. Then, f(x) = x”, where the domain 
of f is the set of all integers, the codomain of f is the set of all integers, and the range of f is the 
set of all integers that are perfect squares, namely, {0, 1, 4, 9, ... }. 


The domain and codomain of functions are often specified in programming languages. For in- 
stance, the Java statement 


int floor(float real){...} 
and the C++ function statement 
int function (float x){...} 


both tell us that the domain of the floor function is the set of real numbers (represented by 
floating point numbers) and its codomain is the set of integers. < 


A function is called real-valued if its codomain is the set of real numbers, and it is called 
integer-valued if its codomain is the set of integers. Two real-valued functions or two integer- 
valued functions with the same domain can be added, as well as multiplied. 


Let f, and fọ be functions from A to R. Then f, +f; and f, fọ are also functions from A to R 
defined for all x € A by 


(A, tH) =f) +A), 
(ff) =f, fr). 


Note that the functions f} +f, and f; fọ have been defined by specifying their values at x in terms 
of the values of f, and fy at x. 


Let f, and f) be functions from R to R such that f,(x) = x? and f,(x) = x — x”. What are the 
functions f, +f, and fi fa? 


Solution: From the definition of the sum and product of functions, it follows that 


tA =fi@ +f) = a + (x - r)y=x 
and 


GRQ =r- =x. 4 
When f is a function from A to B, the image of a subset of A can also be defined. 


Let f be a function from A to B and let S be a subset of A. The image of S under the function 
f is the subset of B that consists of the images of the elements of S. We denote the image of 
S by f(S), so 

F(S) = {t | dsES t = f(s))}. 


We also use the shorthand {f(s) | s E S} to denote this set. 
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EXAMPLE 9 


Remark: The notation f(S) for the image of the set S under the function f is potentially ambigu- 
ous. Here, f(S) denotes a set, and not the value of the function f for the set S. 


Let A = {a,b, c,d, e} and B = {1, 2, 3, 4} with f(a) = 2, f(b) = 1, f(c) = 4, f(d) = 1, and f(e) = 
1. The image of the subset S = {b, c, d} is the set f(S) = {1, 4}. < 


2.3.2 One-to-One and Onto Functions 


Some functions never assign the same value to two different domain elements. These functions 
are said to be one-to-one. 


A function f is said to be one-to-one, or an injection, if and only if f(a) = f(b) implies that 
a = b for all a and b in the domain of f. A function is said to be injective if it is one-to-one. 


Note that a function f is one-to-one if and only if f(a) 4 f(b) whenever a # b. This way of 
expressing that f is one-to-one is obtained by taking the contrapositive of the implication in the 
definition. 


Remark: We can express that f is one-to-one using quantifiers as VaVb( f(a) = f(b) > a = b) 
or equivalently VaVb(a # b > f(a) + f(b)), where the universe of discourse is the domain of 
the function. 


We illustrate this concept by giving examples of functions that are one-to-one and other 
functions that are not one-to-one. 


Determine whether the function f from {a,b,c,d} to {1, 2,3,4,5} with f(a) = 4, f(b) = 5, 
f(c) = 1, and f(d) = 3 is one-to-one. 


Solution: The function f is one-to-one because f takes on different values at the four elements 
of its domain. This is illustrated in Figure 3. < 
Determine whether the function f(x) = x? from the set of integers to the set of integers is one- 


to-one. 


Solution: The function f(x) = x? is not one-to-one because, for instance, f(1) = f(—1) = 1, but 


14-1. 


a® ©] 
be e2 
co e3 
de e4 

e5 


FIGURE 3 A one-to-one function. 
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Remark: The function f(x) = x? with domain Z* is one-to-one. (See the explanation in Example 
12 to see why.) This is a different function from the function in Example 9 because of the 
difference in their domains. 


Determine whether the function f(x) = x + 1 from the set of real numbers to itself is one-to-one. 


Solution: Suppose that x and y are real numbers with f(x) = f(y), so that x + 1 = y+ 1. This 
means that x = y. Hence, f(x) = x + 1 is a one-to-one function from R to R. < 


Suppose that each worker in a group of employees is assigned a job from a set of possible jobs, 
each to be done by a single worker. In this situation, the function f that assigns a job to each 
worker is one-to-one. To see this, note that if x and y are two different workers, then f(x) # f(y) 
because the two workers x and y must be assigned different jobs. < 


We now give some conditions that guarantee that a function is one-to-one. 


A function f whose domain and codomain are subsets of the set of real numbers is called 
increasing if f(x) < f(y), and strictly increasing if f(x) < f(y), whenever x < y and x and y 
are in the domain of f. Similarly, f is called decreasing if f(x) > f(y), and strictly decreasing 
if f(x) > f(y), whenever x < y and x and y are in the domain of f. (The word strictly in this 
definition indicates a strict inequality.) 


Remark: A function f is increasing if VxVy(x < y > f(x) < f()), strictly increasing if VxVy(x < 
y > f(x) < f()), decreasing if VaVy(x < y > f(x) > f(y), and strictly decreasing if VxVy(x < 
y > f(x) > f(y)), where the universe of discourse is the domain of f. 


The function f(x) = x? from Rt to R* is strictly increasing. To see this, suppose that x and y 
are positive real numbers with x < y. Multiplying both sides of this inequality by x gives x? < 
xy. Similarly, multiplying both sides by y gives xy < y*. Hence, f(x) =x? < xy < y? = fO). 
However, the function f(x) = x? from R to the set of nonnegative real numbers is not strictly 
increasing because —1 < 0, but f(—1) = (—1)? = 1 is not less than f(0) = 0? = 0. < 


From these definitions, it can be shown (see Exercises 26 and 27) that a function that is 
either strictly increasing or strictly decreasing must be one-to-one. However, a function that 
is increasing, but not strictly increasing, or decreasing, but not strictly decreasing, is not one- 
to-one. 

For some functions the range and the codomain are equal. That is, every member of the 
codomain is the image of some element of the domain. Functions with this property are called 
onto functions. 


A function f from A to B is called onto, or a surjection, if and only if for every element 
b E€ B there is an element a € A with f(a) = b. A function f is called surjective if it is onto. 


Remark: A function f is onto if Vyax( f(x) = y), where the domain for x is the domain of the 
function and the domain for y is the codomain of the function. 
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FIGURE 4 An onto function. 


We now give examples of onto functions and functions that are not onto. 


Let f be the function from {a, b, c, d} to {1, 2, 3} defined by f(a) = 3, f(b) = 2, f(c) = 1, and 
f(d) = 3. Is f an onto function? 


Solution: Because all three elements of the codomain are images of elements in the domain, we 
see that f is onto. This is illustrated in Figure 4. Note that if the codomain were {1, 2, 3, 4}, then 
f would not be onto. < 


Is the function f(x) = x? from the set of integers to the set of integers onto? 


Solution: The function f is not onto because there is no integer x with x? = —1, for instance. <4 


Is the function f(x) = x + 1 from the set of integers to the set of integers onto? 


Solution: This function is onto, because for every integer y there is an integer x such that f(x) = y. 
To see this, note that f(x) = y if and only if x + 1 = y, which holds if and only if x = y — 1. (Note 
that y — 1 is also an integer, and so, is in the domain of f.) < 


Consider the function f in Example 11 that assigns jobs to workers. The function f is onto if for 
every job there is a worker assigned this job. The function f is not onto when there is at least 
one job that has no worker assigned it. < 


The function f is a one-to-one correspondence, or a bijection, if it is both one-to-one and 
onto. We also say that such a function is bijective. 


Examples 16 and 17 illustrate the concept of a bijection. 


Let f be the function from {a, b, c, d} to {1, 2, 3, 4} with f(a) = 4, f(b) = 2, f(c) = 1, and f(d) = 
3. Is f a bijection? 


Solution: The function f is one-to-one and onto. It is one-to-one because no two values in the do- 
main are assigned the same function value. It is onto because all four elements of the codomain 
are images of elements in the domain. Hence, f is a bijection. < 


Figure 5 displays four functions where the first is one-to-one but not onto, the second is onto 
but not one-to-one, the third is both one-to-one and onto, and the fourth is neither one-to-one 
nor onto. The fifth correspondence in Figure 5 is not a function, because it sends an element to 
two different elements. 


(a) One-to-one, 
not onto 
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(b) Onto, (c) One-to-one (d) Neither one-to-one (e) Nota function 
not one-to-one and onto nor onto 
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FIGURE 5 Examples of different types of correspondences. 
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Suppose that f is a function from a set A to itself. If A is finite, then f is one-to-one if and 
only if it is onto. (This follows from the result in Exercise 74.) This is not necessarily the case 
if A is infinite (as will be shown in Section 2.5). 


Let A be a set. The identity function on A is the function 1, : A > A, where 
u(x) =x 


for all x € A. In other words, the identity function 1, is the function that assigns each element 
to itself. The function 1, is one-to-one and onto, so it is a bijection. (Note that z is the Greek 
letter iota.) < 


For future reference, we summarize what needs be to shown to establish whether a function 
is one-to-one and whether it is onto. It is instructive to review Examples 8-17 in light of this 
summary. 


Suppose that f : A > B. 

To show that f is injective Show that if f(x) = f (y) for arbitrary x, y € A, then x = y. 

To show that f is not injective Find particular elements x, y € A such that x Æ y and f(x) = 
fO). 

To show that f is surjective Consider an arbitrary element y € B and find an element x € A 
such that f(x) = y. 

To show that f is not surjective Find a particular y € B such that f(x) + y for all x € A. 


2.3.3 Inverse Functions and Compositions of Functions 


Now consider a one-to-one correspondence f from the set A to the set B. Because f is an onto 
function, every element of B is the image of some element in A. Furthermore, because f is also 
a one-to-one function, every element of B is the image of a unique element of A. Consequently, 
we can define a new function from B to A that reverses the correspondence given by f. This 
leads to Definition 9. 


Let f be a one-to-one correspondence from the set A to the set B. The inverse function of f is 
the function that assigns to an element b belonging to B the unique element a in A such that 
f(a) = b. The inverse function of f is denoted by f~!. Hence, f7! (b) = a when f(a) = b. 
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FIGURE 6 The function f~! is the inverse of function f. 


Remark: Be sure not to confuse the function f~! with the function 1/f, which is the function 
that assigns to each x in the domain the value 1/f(x). Notice that the latter makes sense only 
when f(x) is a nonzero real number. 


Figure 6 illustrates the concept of an inverse function. 

If a function f is not a one-to-one correspondence, we cannot define an inverse function of 
f. When f is not a one-to-one correspondence, either it is not one-to-one or it is not onto. If f is 
not one-to-one, some element b in the codomain is the image of more than one element in the 
domain. If f is not onto, for some element b in the codomain, no element a in the domain exists 
for which f(a) = b. Consequently, if f is not a one-to-one correspondence, we cannot assign to 
each element b in the codomain a unique element a in the domain such that f(a) = b (because 
for some b there is either more than one such a or no such a). 

A one-to-one correspondence is called invertible because we can define an inverse of this 
function. A function is not invertible if it is not a one-to-one correspondence, because the 
inverse of such a function does not exist. 


Let f be the function from {a, b, c} to {1, 2,3} such that f(a) = 2, f(b) = 3, and f(c) = 1. Is f 
invertible, and if it is, what is its inverse? 


Solution: The function f is invertible because it is a one-to-one correspondence. The in- 
verse function f~! reverses the correspondence given by f, so f-'(1) = c, f7! (2) =a, and f7! 


(3) =b. <l 


Let f : Z > Z be such that f(x) = x + 1. Is f invertible, and if it is, what is its inverse? 


Solution: The function f has an inverse because it is a one-to-one correspondence, as follows 
from Examples 10 and 15. To reverse the correspondence, suppose that y is the image of x, so 
that y = x + 1. Then x = y — 1. This means that y — 1 is the unique element of Z that is sent to 


y by f. Consequently, f! (y) = y — 1. < 
Let f be the function from R to R with f(x) = x’. Is f invertible? 
Solution: Because f(—2) = f (2) = 4, f is not one-to-one. If an inverse function were defined, it 


would have to assign two elements to 4. Hence, f is not invertible. (Note we can also show that 
f is not invertible because it is not onto.) < 


Sometimes we can restrict the domain or the codomain of a function, or both, to obtain an 
invertible function, as Example 22 illustrates. 


EXAMPLE 22 
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Show that if we restrict the function f(x) = x? in Example 21 to a function from the set of all 
nonnegative real numbers to the set of all nonnegative real numbers, then f is invertible. 


Solution: The function f(x) = x? from the set of nonnegative real numbers to the set of non- 
negative real numbers is one-to-one. To see this, note that if f(x) = f(y), then x? = y’, so 
xX — y? =(x+y)(x — y) = 0. This means that x + y = 0 or x — y = 0, sox = —y or x = y. Be- 
cause both x and y are nonnegative, we must have x = y. So, this function is one-to-one. 
Furthermore, f(x) = x? is onto when the codomain is the set of all nonnegative real numbers, be- 
cause each nonnegative real number has a square root. That is, if y is a nonnegative real number, 
there exists a nonnegative real number x such that x = Jy: which means that x” = y. Because 
the function f(x) = x? from the set of nonnegative real numbers to the set of nonnegative real 
numbers is one-to-one and onto, it is invertible. Its inverse is given by the rule f—!(y) = y5- < 


Let g be a function from the set A to the set B and let f be a function from the set B to the 
set C. The composition of the functions f and g, denoted for all a € A by fog, is the function 
from A to C defined by 


(fog)(a) = f(&la)). 


In other words, fog is the function that assigns to the element a of A the element assigned by 
f to g(a). The domain of fog is the domain of g. The range of fog is the image of the range 
of g with respect to the function f. That is, to find (fog)(a) we first apply the function g to a 
to obtain g(a) and then we apply the function f to the result g(a) to obtain (fog)(a) = f (e(a)). 
Note that the composition fog cannot be defined unless the range of g is a subset of the domain 
of f. In Figure 7 the composition of functions is shown. 


Let g be the function from the set {a, b, c} to itself such that g(a) = b, g(b) = c, and g(c) = a. 
Let f be the function from the set {a, b, c} to the set {1, 2, 3} such that f(a) = 3, f(b) = 2, and 
f(c) = 1. What is the composition of f and g, and what is the composition of g and f? 


Solution: The composition fog is defined by (fog)(a) = f(e(a)) = f(b) = 2, (fog) (b) = 
FE) = f(c) = 1, and (fog)(c) = f(s(e)) = fa) = 3. 

Note that gof is not defined, because the range of f is not a subset of the domain 
of g. < 


(foga) 


FIGURE 7 The composition of the functions f and g. 
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EXAMPLE 24 


EXAMPLE 25 


Definition 11 


Let f and g be the functions from the set of integers to the set of integers defined by 
f(x) = 2x +3 and g(x) = 3x + 2. What is the composition of f and g? What is the composition 
of g and f? 


Solution: Both the compositions fog and gof are defined. Moreover, 
(fog)(x) = f(g) =f(3x + 2) = 23x +2) +3 = 6x+7 
and 


(gof)(x) = g(f(x)) = g(2x + 3) = 3(2x +3) 4+2=6x4 11. 4 


Remark: Note that even though fog and gof are defined for the functions f and g in Example 24, 
fog and gof are not equal. In other words, the commutative law does not hold for the composition 
of functions. 


Let f and g be the functions defined by f : R > R*+ u {0} with f(x) = x? and g : RtU{0} > R 
with g(x) = yx (where yx is the nonnegative square root of x). What is the function (fog)(x)? 


Solution: The domain of (fog)(x) = f(g(x)) is the domain of g, whichis R* U {0}, the set of non- 

negative real numbers. If x is a nonnegative real number, we have (fog)(x) = f(g(x)) = f( 4/x) = 
2 

(vx ) = x. The range of fog is the image of the range of g with respect to the function f. This is 


the set Rt u {0}, the set of nonnegative real numbers. Summarizing, f : R* U {0} > Rt u {0} 
and f(g(x)) = x for all x. < 


When the composition of a function and its inverse is formed, in either order, an identity 
function is obtained. To see this, suppose that f is a one-to-one correspondence from the set A 
to the set B. Then the inverse function f~! exists and is a one-to-one correspondence from B 
to A. The inverse function reverses the correspondence of the original function, so f~!(b) = a 
when f(a) = b, and f(a) = b when f~!(b) = a. Hence, 


(f lof Ya) =F) =f") = a, 
and 
(fof ')(b) =f) =f@ =b. 
Consequently f~!of = 1, and fof! = 1,, where 1, and 1, are the identity functions on the sets 


A and B, respectively. That is, (fD! = f. 


2.3.4 The Graphs of Functions 


We can associate a set of pairs in A x B to each function from A to B. This set of pairs is called 
the graph of the function and is often displayed pictorially to aid in understanding the behavior 
of the function. 


Let f be a function from the set A to the set B. The graph of the function f is the set of ordered 
pairs {(a, b) | a E A and f (a) = b}. 


EXAMPLE 26 


EXAMPLE 27 


Definition 12 
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From the definition, the graph of a function f from A to B is the subset of A x B containing the 
ordered pairs with the second entry equal to the element of B assigned by f to the first entry. 
Also, note that the graph of a function f from A to B is the same as the relation from A to B 
determined by the function f, as described on Section 2.3.1. 


Display the graph of the function f(n) = 2n + 1 from the set of integers to the set of integers. 


Solution: The graph of f is the set of ordered pairs of the form (n, 2n + 1), where n is an integer. 
This graph is displayed in Figure 8. < 


Display the graph of the function f(x) = x? from the set of integers to the set of integers. 


Solution: The graph of f is the set of ordered pairs of the form (x, f(x)) = (x, x”), where x is an 
integer. This graph is displayed in Figure 9. < 


2.3.5 Some Important Functions 


Next, we introduce two important functions in discrete mathematics, namely, the floor and ceil- 
ing functions. Let x be a real number. The floor function rounds x down to the closest integer 
less than or equal to x, and the ceiling function rounds x up to the closest integer greater than 
or equal to x. These functions are often used when objects are counted. They play an important 
role in the analysis of the number of steps used by procedures to solve problems of a particular 
size. 


The floor function assigns to the real number x the largest integer that is less than or equal to 
x. The value of the floor function at x is denoted by |x]. The ceiling function assigns to the 
real number x the smallest integer that is greater than or equal to x. The value of the ceiling 
function at x is denoted by [x]. 


Remark: The floor function is often also called the greatest integer function. It is often denoted 
by [x]. 


(0) (0) Q (0) e (0) [e] (0) e (-3,9) (3,9)0 


o 6 © © © © -© 6 oe) 2,4)0 
O0—O0—0 p-p- 0 0 ő 
a 2 © © © <6 GS ð (lle e(1,1) 
ð o @ © © 0 @ 6 (0.0) 
FIGURE 8 The graph of FIGURE 9 The graph of f(x) = x? 


f(n) = 2n +1 from Z to Z. from Z to Z. 
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EXAMPLE 28 


Links » 


EXAMPLE 29 


EXAMPLE 30 
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(a) y= |x] (b) y= [x] 


FIGURE 10 Graphs of the (a) floor and (b) ceiling functions. 


These are some values of the floor and ceiling functions: 


5) =0, [$] =1, 1-3) =-L I-31 = 0, [3.1] = 3, [3.1] =4 |7] =7 [7] =7. 4 


We display the graphs of the floor and ceiling functions in Figure 10. In Figure 10(a) we display 
the graph of the floor function |x]. Note that this function has the same value throughout the 
interval [n,n + 1), namely n, and then it jumps up to n+ 1 when x =n +1. In Figure 10(b) 
we display the graph of the ceiling function [x]. Note that this function has the same value 
throughout the interval (n, n + 1], namely n + 1, and then jumps to n + 2 when x is a little larger 
than n + 1. 

The floor and ceiling functions are useful in a wide variety of applications, including those 
involving data storage and data transmission. Consider Examples 29 and 30, typical of basic 
calculations done when database and data communications problems are studied. 


Data stored on a computer disk or transmitted over a data network are usually represented as a 
string of bytes. Each byte is made up of 8 bits. How many bytes are required to encode 100 bits 
of data? 


Solution: To determine the number of bytes needed, we determine the smallest integer that is at 
least as large as the quotient when 100 is divided by 8, the number of bits in a byte. Consequently, 
[100/8] = [12.5] = 13 bytes are required. < 


In asynchronous transfer mode (ATM) (a communications protocol used on backbone net- 
works), data are organized into cells of 53 bytes. How many ATM cells can be transmitted 
in 1 minute over a connection that transmits data at the rate of 500 kilobits per second? 


Solution: In 1 minute, this connection can transmit 500,000 - 60 = 30,000,000 bits. Each ATM 
cell is 53 bytes long, which means that it is 53 - 8 = 424 bits long. To determine the number 
of cells that can be transmitted in 1 minute, we determine the largest integer not exceeding the 
quotient when 30,000,000 is divided by 424. Consequently, |30,000,000/424| = 70,754 ATM 
cells can be transmitted in 1 minute over a 500 kilobit per second connection. < 


Table 1, with x denoting a real number, displays some simple but important properties 
of the floor and ceiling functions. Because these functions appear so frequently in discrete 


EXAMPLE 31 


Extra » 
Examples 


Links > 
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TABLE 1 Useful Properties of the 
Floor and Ceiling Functions. 
(n is an integer, x is a real number) 


(la) |x] =n if and only ifn<x<n+1 
(1b) [x] =nifandonlyifn-—l<x<n 
(Ic) |x] =nif and only ifx-l<n<x 
(1d) [x] =nifand only ifx<n<x+1 


(2) x-1< |x] <x< [x] <x4+1 


Ga) [=x] = -[a] 
Gb) [-x] =-lx] 


(4a) |x+n] = |x] +n 
(4b) [x+n] = [x] +n 


mathematics, it is useful to look over these identities. Each property in this table can be es- 
tablished using the definitions of the floor and ceiling functions. Properties (la), (1b), (1c), and 
(1d) follow directly from these definitions. For example, (1a) states that |x] = n if and only if 
the integer n is less than or equal to x and n + 1 is larger than x. This is precisely what it means 
for n to be the greatest integer not exceeding x, which is the definition of |x] = n. Properties 
(1b), (1c), and (1d) can be established similarly. We will prove property (4a) using a direct proof. 


Proof: Suppose that |x] = m, where m is a positive integer. By property (1a), it follows that 
m<x<m-+1. Adding n to all three quantities in this chain of two inequalities shows that 
m+n<x+n<m+n+ 1. Using property (la) again, we see that |x +n] =m+n= |x] +n. 
This completes the proof. Proofs of the other properties are left as exercises. 


The floor and ceiling functions enjoy many other useful properties besides those displayed 
in Table 1. There are also many statements about these functions that may appear to be cor- 
rect, but actually are not. We will consider statements about the floor and ceiling functions in 
Examples 31 and 32. 

A useful approach for considering statements about the floor function is to let x =n + e, 
where n = |x] is an integer, and e, the fractional part of x, satisfies the inequality O < e < 1. 
Similarly, when considering statements about the ceiling function, it is useful to write x = n — e, 
where n = [x] is an integer and O < e < 1. 


Prove that if x is a real number, then |2x] = [x] + [x+ J. 


Solution: To prove this statement we let x = n + e, where n is an integer and O < e < 1. There 
are two cases to consider, depending on whether e is less than, or greater than or equal to L. 
(The reason we choose these two cases will be made clear in the proof.) 


We first consider the case when 0 < € < L. In this case, 2x = 2n + 2e and [2x] = 2n be- 
cause 0 < 2e < 1. Similarly, x + L =n+ G +€), so |x + al = n, because 0 < L +e < 1. Con- 
sequently, [2x] = 2n and |x] + |x + al =n+n=2n. 

Next, we consider the case when = <e<1. In this case, 2x= 2n + 2e = 
(2n+1)+(2e-1). Because 0<2e-1<1, it follows that [2x] =2n+1. Because 


160 2/Basic Structures: Sets, Functions, Sequences, Sums, and Matrices 


[x+ 5] = [n+ G +e)]=|n+1+(e- Ż)] and O<e-5<1, it follows that 


lx+ `] =n+ 1. Consequently, [2x] =2n+1 and |x] +|x+ 1] =n+(n+1)=2n+1. 
This concludes the proof. < 


EXAMPLE 32 Prove or disprove that [x + y] = [x] + [y] for all real numbers x and y. 


Solution: Although this statement may appear reasonable, it is false. A counterexample is sup- 
plied by x = ; and y= L. With these values we find that [x+y] = É + I = [1] = 1, but 


[x] + [yl = [$1 + [$1 =141=2. < 


There are certain types of functions that will be used throughout the text. These include 
polynomial, logarithmic, and exponential functions. A brief review of the properties of these 
functions needed in this text is given in Appendix 2. In this book the notation log x will be used 
to denote the logarithm to the base 2 of x, because 2 is the base that we will usually use for 
logarithms. We will denote logarithms to the base b, where b is any real number greater than 1, 
by log, x, and the natural logarithm by In x. 

Another function we will use throughout this text is the factorial function f: N > Z+, 
denoted by f(n) = n!. The value of f(n) = n! is the product of the first n positive integers, so 
f(a) =1-2+-(@-—1)-n[and f() = 0! = 1). 


EXAMPLE 33 We have f(1) = 1! = 1, f(2) = 2! =1-2 =2, f(6) = 6! = 1-2-3-4-5-6 = 720, 
and f(20) = 1-2-3-4-5-6-7-8-9-10-11-12-13-14-15-16-17-18-19-20 = 
2,432,902,008, 176,640,000. < 


Example 33 illustrates that the factorial function grows extremely rapidly as n grows. 
The rapid growth of the factorial function is made clearer by Stirling’s formula, a result 


from higher mathematics that tells us that n! ~ ~/2an(n/e)”". Here, we have used the notation 
f(n) ~ g(n), which means that the ratio f(n)/g(n) approaches 1 as n grows without bound (that 
is, lim,_,,./()/g(n) = 1). The symbol ~ is read “is asymptotic to.” Stirling’s formula is named 


n—=> CO 


after James Stirling, a Scottish mathematician of the eighteenth century. 


JAMES STIRLING (1692-1770) James Stirling was born near the town of Stirling, Scotland. His family strongly supported the 
Jacobite cause of the Stuarts as an alternative to the British crown. The first information known about James is that he entered Balliol 
College, Oxford, on a scholarship in 1711. However, he later lost his scholarship when he refused to pledge his allegiance to the British 
crown. The first Jacobean rebellion took place in 1715, and Stirling was accused of communicating with rebels. He was charged with 
cursing King George, but he was acquitted of these charges. Even though he could not graduate from Oxford because of his politics, he 
remained there for several years. Stirling published his first work, which extended Newton’s work on plane curves, in 1717. He traveled 
to Venice, where a chair of mathematics had been promised to him, an appointment that unfortunately fell through. Nevertheless, 
Stirling stayed in Venice, continuing his mathematical work. He attended the University of Padua in 1721, and in 1722 he returned to 
Glasgow. Stirling apparently fled Italy after learning the secrets of the Italian glass industry, avoiding the efforts of Italian glass makers 
to assassinate him to protect their secrets. 

In late 1724 Stirling moved to London, staying there 10 years teaching mathematics and actively engaging in research. In 1730 
he published Methodus Differentialis, his most important work, presenting results on infinite series, summations, interpolation, and 
quadrature. It is in this book that his asymptotic formula for n! appears. Stirling also worked on gravitation and the shape of the earth; 
he stated, but did not prove, that the earth is an oblate spheroid. Stirling returned to Scotland in 1735, when he was appointed manager 
of a Scottish mining company. He was very successful in this role and even published a paper on the ventilation of mine shafts. He 
continued his mathematical research, but at a reduced pace, during his years in the mining industry. Stirling is also noted for surveying 
the River Clyde with the goal of creating a series of locks to make it navigable. In 1752 the citizens of Glasgow presented him with a 
silver teakettle as a reward for this work. 
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Exercises 


1. Why is f not a function from R to R if 


a) f(x) = 1/x? 
b) f@) = yx? 


o) fQ =+V0?+4+ 1)? 


2. Determine whether f is a function from Z to R if 


a) f(n) = +n. 
b) f(n) = 


yn? +1. 
©) f) = 1/0 - 4). 

3. Determine whether f is a function from the set of all bit 
strings to the set of integers if 
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2.3.6 Partial Functions 


A program designed to evaluate a function may not produce the correct value of the function for 
all elements in the domain of this function. For example, a program may not produce a correct 
value because evaluating the function may lead to an infinite loop or an overflow. Similarly, in 
abstract mathematics, we often want to discuss functions that are defined only for a subset of 
the real numbers, such as 1/x, /x, and arcsin (x). Also, we may want to use such notions as the 
“youngest child” function, which is undefined for a couple having no children, or the “time of 
sunrise,” which is undefined for some days above the Arctic Circle. To study such situations, 
we use the concept of a partial function. 


A partial function f from a set A to a set B is an assignment to each element a in a subset 
of A, called the domain of definition of f, of a unique element b in B. The sets A and B are 
called the domain and codomain of f, respectively. We say that f is undefined for elements 
in A that are not in the domain of definition of f. When the domain of definition of f equals 
A, we say that f is a total function. 


Remark: We write f : A > B to denote that f is a partial function from A to B. Note that this is 
the same notation as is used for functions. The context in which the notation is used determines 
whether f is a partial function or a total function. 


The function f : Z > R where f(n) = s/n is a partial function from Z to R where the domain of 
definition is the set of nonnegative integers. Note that f is undefined for negative integers. <4 


5. Find the domain and range of these functions. Note that 
in each case, to find the domain, determine the set of el- 
ements assigned values by the function. 

a) the function that assigns to each bit string the number 
of ones in the string minus the number of zeros in the 
string 

b) the function that assigns to each bit string twice the 
number of zeros in that string 


c) the function that assigns the number of bits left over 
when a bit string is split into bytes (which are blocks 


a) f(S) is the position of a 0 bit in S. 

b) f(S) is the number of 1 bits in S. 

c) f(S) is the smallest integer i such that the ith bit of S is 
1 and f(S) = 0 when S is the empty string, the string 
with no bits. 


4. Find the domain and range of these functions. Note that 


in each case, to find the domain, determine the set of el- 
ements assigned values by the function. 


a) the function that assigns to each nonnegative integer 
its last digit 

b) the function that assigns the next largest integer to a 
positive integer 

c) the function that assigns to a bit string the number of 
one bits in the string 

d) the function that assigns to a bit string the number of 
bits in the string 


of 8 bits) 


d) the function that assigns to each positive integer the 
largest perfect square not exceeding this integer 


6. Find the domain and range of these functions. 


a) the function that assigns to each pair of positive inte- 
gers the first integer of the pair 


b) the function that assigns to each positive integer its 
largest decimal digit 

c) the function that assigns to a bit string the number of 
ones minus the number of zeros in the string 


d) the function that assigns to each positive integer the 
largest integer not exceeding the square root of the in- 
teger 

e) the function that assigns to a bit string the longest 
string of ones in the string 
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7. 


10. 


11. 
12. 


13. 
14. 


15. 


16. 


Find the domain and range of these functions. 

a) the function that assigns to each pair of positive inte- 
gers the maximum of these two integers 

b) the function that assigns to each positive integer the 
number of the digits 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 that do 
not appear as decimal digits of the integer 

c) the function that assigns to a bit string the number of 
times the block 11 appears 

d) the function that assigns to a bit string the numerical 
position of the first 1 in the string and that assigns the 
value 0 to a bit string consisting of all Os 


. Find these values. 


a) [1.1] b) [1.1] 

c) [-0.1] d) [-0.1] 

e) [2.99] £) [-2.99] 

g& li +f] h) [Li] + [i1 +] 
. Find these values. 

a) [=] b) [2] 

© [-3] d) |-2 

e) [3] f) |-1] 

g +931] h) L- Lž]] 


Determine whether each of these functions from 
{a, b, c, d} to itself is one-to-one. 


a) f(a) = b, f(b) = a, f(c) = c, f(d) = d 
b) f(a) = b, f(b) = b, f(c) = d, f(d) = c 
c) f(a) = d, f(b) = b, f(c) = c, f(d) = d 


Which functions in Exercise 10 are onto? 

Determine whether each of these functions from Z to Z 
is one-to-one. 

a) f(n)=n-1 b) f(n)=n? +1 

9 fn =n d) f(n) = [n/2] 

Which functions in Exercise 12 are onto? 

Determine whether f: Z x Z > Z is onto if 

a) f(m,n) = 2m—n. 

b) f(m, n) =m? -= n. 

c) f(m,n)=m+n+1. 

d) f(m, n) = |m] — |n]. 

e) f(m,n) =m? — 4. 

Determine whether the function f: Z x Z > Z is onto if 
a) f(m, n)=m+n. 

b) f(m,n) =m +r. 


c) f(m,n) =m. 
d) fm, n) = jnl]. 
e) f(m,n) =m-—n. 


Consider these functions from the set of students in a 
discrete mathematics class. Under what conditions is the 
function one-to-one if it assigns to a student his or her 
a) mobile phone number. 

b) student identification number. 

c) final grade in the class. 

d) home town. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


27. 


28. 


Consider these functions from the set of teachers in a 

school. Under what conditions is the function one-to-one 

if it assigns to a teacher his or her 

a) office. 

b) assigned bus to chaperone in a group of buses taking 
students on a field trip. 

c) salary. 

d) social security number. 

Specify a codomain for each of the functions in Exercise 

16. Under what conditions is each of these functions with 

the codomain you specified onto? 

Specify a codomain for each of the functions in Exercise 

17. Under what conditions is each of the functions with 

the codomain you specified onto? 

Give an example of a function from N to N that is 

a) one-to-one but not onto. 

b) onto but not one-to-one. 

c) both onto and one-to-one (but different from the iden- 
tity function). 

d) neither one-to-one nor onto. 

Give an explicit formula for a function from the set of 

integers to the set of positive integers that is 

a) one-to-one, but not onto. 

b) onto, but not one-to-one. 

c) one-to-one and onto. 

d) neither one-to-one nor onto. 

Determine whether each of these functions is a bijection 

from R to R. 

a) f(x) = —3x + 4 

b) f(x) = -3x +7 

c) fœ = (x + 1)/@ +2) 

d) f= +1 

Determine whether each of these functions is a bijection 

from R to R. 

a) fx)=2x+1 

b) f@®) =x +1 

c) f@=xr 

d) f® =P +D +2) 

Let f: R > R and let f(x) > 0 for all x € R. Show that 

F(x) is strictly increasing if and only if the function g(x) = 

1/f() is strictly decreasing. 

Let f: R > R and let f(x) > 0 for all x € R. Show that 

J (x) is strictly decreasing if and only if the function g(x) = 

1/f(x) is strictly increasing. 

a) Prove that a strictly increasing function from R to it- 
self is one-to-one. 

b) Give an example of an increasing function from R to 
itself that is not one-to-one. 

a) Prove that a strictly decreasing function from R to it- 
self is one-to-one. 

b) Give an example of a decreasing function from R to 
itself that is not one-to-one. 

Show that the function f(x) = e* from the set of real num- 

bers to the set of real numbers is not invertible, but if the 

codomain is restricted to the set of positive real numbers, 

the resulting function is invertible. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


*36. 


*37. 


38. 


39. 


40. 


41. 


42. 


43. 


Show that the function f(x) = |x| from the set of real 
numbers to the set of nonnegative real numbers is not in- 
vertible, but if the domain is restricted to the set of non- 
negative real numbers, the resulting function is invertible. 


Let S = {—1, 0, 2, 4, 7}. Find f(S) if 

a) f@®=1. b) f(@) = 2x41. 

c) fœ) = [x/5]. d) fx) =L? + 1/3]. 
Let f(x) = |x?/3]. Find f(S) if 


a) S = {-2, —1, 0, 1, 2, 3}. 

b) S= {0, 1, 2, 3,4, 5}. 

c) S={1,5,7, 11}. 

d) S= {2, 6, 10, 14}. 

Let f(x) = 2x where the domain is the set of real numbers. 
What is 

a) f(Z)? b) f(N)? c) f(R)? 

Suppose that g is a function from A to B and f is a func- 
tion from B to C. 


a) Show that if both f and g are one-to-one functions, 
then fog is also one-to-one. 


b) Show that if both f and g are onto functions, then fog 
is also onto. 


Suppose that g is a function from A to B and f is a func- 

tion from B to C. Prove each of these statements. 

a) If fog is onto, then f must also be onto. 

b) If fog is one-to-one, then g must also be one-to-one. 

c) If fog is a bijection, then g is onto if and only if f is 
one-to-one. 

Find an example of functions f and g such that fog is a 

bijection, but g is not onto and f is not one-to-one. 


If f and fog are one-to-one, does it follow that g is one- 
to-one? Justify your answer. 


If f and fog are onto, does it follow that g is onto? Justify 
your answer. 


Find fog and gof, where f(x) = x? + 1 and g(x) =x +2, 
are functions from R to R. 


Find f + g and fg for the functions f and g given in Exer- 
cise 36. 


Let f(x) = ax + b and g(x) = cx + d, where a, b, c, and d 
are constants. Determine necessary and sufficient condi- 
tions on the constants a, b, c, and d so that fog = gof. 


Show that the function f(x) = ax + b from R to R, where 
a and b are constants with a # 0 is invertible, and find the 
inverse of f. 


Let f be a function from the set A to the set B. Let S and 
T be subsets of A. Show that 

a) ASUT) =f(S) Uf(7). 

b) ASAT) CFS) Nf). 


a) Give an example to show that the inclusion in part (b) 
in Exercise 42 may be proper. 
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b) Show that if f is one-to-one, the inclusion in part (b) 
in Exercise 42 is an equality. 
Let f be a function from the set A to the set B. Let S be a 
subset of B. We define the inverse image of S to be the sub- 
set of A whose elements are precisely all preimages of all 
elements of S$. We denote the inverse image of S by f~!(S), 
so f—!(S) = {a E€ A | f(a) € S}. [Beware: The notation f~! is 
used in two different ways. Do not confuse the notation intro- 
duced here with the notation f—!(y) for the value at y of the 
inverse of the invertible function f. Notice also that f~! (S), 
the inverse image of the set S, makes sense for all functions f, 
not just invertible functions.] 
44. Let f be the function from R to R defined by 
f(x) = x*. Find 
a) FAL). b) f'({x|0<x<1)}). 
c) f(x | x> 4}). 
45. Let g(x) = |x]. Find 
a) g-'({0}). b) g*({-1,0,1}). 
c) gl({x|0<x<1}). 
46. Let f be a function from A to B. Let S and T be subsets of 
B. Show that 
a) f (SUT) =f (S) Uf (7). 
b) f'(SNT) =f (Saf T). 
47. Let f be a function from A to B. Let S be a subset of B. 
Show that f~! (S) = f-1($). 
48. Show that |x + 2] is the closest integer to the number x, 
except when x is midway between two integers, when it 
is the larger of these two integers. 


49. Show that [x — z] is the closest integer to the number x, 
except when x is midway between two integers, when it 
is the smaller of these two integers. 

50. Show that if x is a real number, then [x] — |x] = 1 if xis 
not an integer and [x] — |x] = 0 if x is an integer. 

51. Show that if x is a real number, then x — 1 < |x] <x < 
[x] <x+1. 

52. Show that if x is a real number and m is an integer, then 
[x+m] = [x] +m. 

53. Show that if x is a real number and n is an integer, then 
a) x < nif and only if |x] <n. 

b) n <x if and only ifn < [x]. 


54. Show that if x is a real number and n is an integer, then 
a) x < nif and only if [x] <n. 
b) n <xif and only ifn < [x]. 


55. Prove that if n is an integer, then |n/2] = n/2ifn is even 
and (n — 1)/2 if nis odd. 

56. Prove that if x is a real number, then |—x] = —[x] and 
[-x] =-|x]. 

57. The function INT is found on some calculators, where 
INT(x) = |x] when x is a nonnegative real number and 
INT(x) = [x] when x is a negative real number. Show 
that this INT function satisfies the identity INT(—x) = 
—INT(a). 
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58. 


59. 


60. 


61. 


62. 


63. 


64. 
65. 


66. 
67. 


68. 


69. 


70. 


71. 
72. 


73. 


Let a and b be real numbers with a < b. Use the floor 
and/or ceiling functions to express the number of inte- 
gers n that satisfy the inequality a < n < b. 

Let a and b be real numbers with a < b. Use the floor 
and/or ceiling functions to express the number of inte- 
gers n that satisfy the inequality a <n < b. 

How many bytes are required to encode n bits of data 
where n equals 

a) 4? b) 10? c) 500? d) 3000? 

How many bytes are required to encode n bits of data 
where n equals 

a) 7? b) 17? c) 1001? d) 28,800? 
How many ATM cells (described in Example 30) can be 
transmitted in 10 seconds over a link operating at the fol- 
lowing rates? 

a) 128 kilobits per second (1 kilobit = 1000 bits) 

b) 300 kilobits per second 

c) 1 megabit per second (1 megabit = 1,000,000 bits) 
Data are transmitted over a particular Ethernet network 
in blocks of 1500 octets (blocks of 8 bits). How many 
blocks are required to transmit the following amounts of 
data over this Ethernet network? (Note that a byte is a 
synonym for an octet, a kilobyte is 1000 bytes, and a 
megabyte is 1,000,000 bytes.) 

a) 150 kilobytes of data 

b) 384 kilobytes of data 

c) 1.544 megabytes of data 

d) 45.3 megabytes of data 

Draw the graph of the function f(n) = 1 — n? from Z to Z. 
Draw the graph of the function f(x) = [2x] from R 
toR. 

Draw the graph of the function f(x) = |x/2] from R to R. 
Draw the graph of the function f(x) = |x] + [x/2] from 
RtoR. 

Draw the graph of the function f(x) = [x] + |x/2] from 
RtoR. 

Draw graphs of each of these functions. 


a) f(x) = Lt 5] b) f(x) = [2x +1) 


© f(x) = [x/3] d) f(x) = [1/2] 
e f(x) = [x-2] + [x42] ; ; 
g) fix) = fle- t +4] 


£) fœ = [2x] [x/2] 


Draw graphs of each of these functions. 


a) f(x) = [3x -2] b) fœ) = [0.2x] 
c) f(x) = |-1/x] d f(x) = r] 
e) f(x) = [x/2]|x/2] £) f@) = [x/2] + [x/2] 


g f@) = [2 [2/2] + 5] 

Find the inverse function of f(x) = x? + 1. 

Suppose that f is an invertible function from Y to Z and 
g is an invertible function from X to Y. Show that the 
inverse of the composition fog is given by (fog)! = 
a ee 

Let S be a subset of a universal set U. The character- 
istic function f; of S is the function from U to the set 
{0, 1} such that f,(x) = 1 if x belongs to S and f;(x) = 0 
if x does not belong to S. Let A and B be sets. Show that 
for all x € U, 


cr 74, 


75. 


76. 


77. 


78. 


79. 


80. 


bs" 81. 


* 82, 


a) fara) = fa) - fg) 

b) fara) = fa + fB — fa) «fed 

©) EWL RA 

d) Saws) = fy) + fg) — fa fg) 

Suppose that f is a function from A to B, where A and B 

are finite sets with |A| = |B|. Show that f is one-to-one if 

and only if it is onto. 

Prove or disprove each of these statements about the floor 

and ceiling functions. 

a) [|x]] = |x] for all real numbers x. 

b) [2x] = 2|x] whenever x is a real number. 

c) [x] + [y] — [x+y] =0 or 1 whenever x and y are 
real numbers. 


d) [xy] = [x] [y] for all real numbers x and y. 

e) xX] _|xt 
2 

Prove or disprove each of these statements about the floor 


and ceiling functions. 


for all real numbers x. 


a) | [x] ] = [x] for all real numbers x. 

b) [x+y] = Lx] + Ly] for all real numbers x and y. 

c) [ [x/2] /2] = [x/4] for all real numbers x. 

d) VARS ]= Lyx] for all positive real numbers x. 

e) [x] + ly] + [x+y] < [2x] + [2y] for all real num- 
bers x and y. 

Prove that if x is a positive real number, then 


a) LVL] = Lyx]. 

b) [yx] = [vx]. 

Let x be a real number. 

[x] + [e+ =] + [e+ 2]. 

For each of these partial functions, determine its domain, 

codomain, domain of definition, and the set of values for 

which it is undefined. Also, determine whether it is a total 

function. 

a) f: Z > R, f(y) =1/n 

b) f: Z > Z, f(n) = [n/2] 

c) f: ZXZ > Q, f(m, n)=m/n 

d) f: Z xZ > Z, f(m, n) = mn 

e) f:ZxZ >Z, f(m n)=m-nifm>n 

a) Show thata partial function from A to B can be viewed 
as a function f* from A to B U {u}, where u is not an 
element of B and 


Show that [3x] = 


f(a) if a belongs to the domain 
of definition of f 
u iff is undefined at a. 


Fa = 


b) Using the construction in (a), find the function f* cor- 
responding to each partial function in Exercise 79. 

a) Show that if a set § has cardinality m, where m is a 
positive integer, then there is a one-to-one correspon- 
dence between S and the set {1, 2, ..., m}. 

b) Show that if S and T are two sets each with m ele- 
ments, where m is a positive integer, then there is a 
one-to-one correspondence between S and T. 

Show that a set S is infinite if and only if there is a proper 

subset A of S such that there is a one-to-one correspon- 

dence between A and S. 
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Sequences and Summations 


Definition 1 


EXAMPLE 1 


2.4.1 Introduction 


Sequences are ordered lists of elements, used in discrete mathematics in many ways. For ex- 
ample, they can be used to represent solutions to certain counting problems, as we will see in 
Chapter 8. They are also an important data structure in computer science. We will often need 
to work with sums of terms of sequences in our study of discrete mathematics. This section 
reviews the use of summation notation, basic properties of summations, and formulas for the 
sums of terms of some particular types of sequences. 

The terms of a sequence can be specified by providing a formula for each term of the se- 
quence. In this section we describe another way to specify the terms of a sequence using a 
recurrence relation, which expresses each term as a combination of the previous terms. We will 
introduce one method, known as iteration, for finding a closed formula for the terms of a se- 
quence specified via a recurrence relation. Identifying a sequence when the first few terms are 
provided is a useful skill when solving problems in discrete mathematics. We will provide some 
tips, including a useful tool on the Web, for doing so. 


2.4.2 Sequences 


A sequence is a discrete structure used to represent an ordered list. For example, 1, 2, 3, 5, 8 is 
a sequence with five terms and 1, 3,9, 27, 81, ... , 3”, ... is an infinite sequence. 


A sequence is a function from a subset of the set of integers (usually either the set {0, 1, 2, ...} 
or the set {1, 2, 3, ...}) toa set S. We use the notation a, to denote the image of the integer n. 
We call a,, a term of the sequence. 


We use the notation {a,,} to describe the sequence. (Note that a, represents an individual 
term of the sequence {a,,}. Be aware that the notation {a,,} for a sequence conflicts with the 
notation for a set. However, the context in which we use this notation will always make it clear 
when we are dealing with sets and when we are dealing with sequences. Moreover, although we 
have used the letter a in the notation for a sequence, other letters or expressions may be used 
depending on the sequence under consideration. That is, the choice of the letter a is arbitrary.) 

We describe sequences by listing the terms of the sequence in order of increasing subscripts. 


Consider the sequence {a,,}, where 


The list of the terms of this sequence, beginning with a,, namely, 
a), 4), a3, Ag, «++, 
starts with 


ice 
ia Ww aaa a 
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Definition 2 


EXAMPLE 2 


Definition 3 


EXAMPLE 3 


EXAMPLE 4 


A geometric progression is a sequence of the form 
n 


a, ar, ar?, ..., AT”, ... 


where the initial term a and the common ratio r are real numbers. 


Remark: A geometric progression is a discrete analogue of the exponential function f(x) = ar’. 

The sequences {b,,} with b, = (—1)", {c,} with c, = 2-5”, and {d,} with d, = 6- (1/3)” are 

geometric progressions with initial term and common ratio equal to 1 and —1, 2 and 5, and 6 

and 1/3, respectively, if we start at n = 0. The list of terms bọ, by, b>, b3, by, ... begins with 
1,-L1,-1,1...; 

the list of terms Cp, C1, C2, C3, C4, ... begins with 


2, 10, 50, 250, 1250, ... ; 


and the list of terms dọ, d4, dz, d3, dy, ... begins with 


An arithmetic progression is a sequence of the form 
aa+d,a+2d,...,a+nd,... 


where the initial term a and the common difference d are real numbers. 


Remark: An arithmetic progression is a discrete analogue of the linear function f(x) = dx + a. 


The sequences {s,} with s, = —1 + 4n and {t,} with t, = 7 — 3n are both arithmetic progres- 
sions with initial terms and common differences equal to —1 and 4, and 7 and —3, respectively, 
if we start at n = 0. The list of terms so, 5, 5, 53, ... begins with 


=1,.37 7, 11,4253 
and the list of terms tọ, t, t», tz, ... begins with 


7,4, 1, —2,.... < 


Sequences of the form a}, a5, ..., a, are often used in computer science. These finite se- 
quences are also called strings. This string is also denoted by aa, ... a„. (Recall that bit strings, 
which are finite sequences of bits, were introduced in Section 1.1.) The length of a string is the 
number of terms in this string. The empty string, denoted by 4, is the string that has no terms. 
The empty string has length zero. 


The string abcd is a string of length four. < 


Definition 4 


EXAMPLE 5 


EXAMPLE 6 


Hop along to Chapter 8 
to learn how to find a 
formula for the 
Fibonacci numbers. 


Links » 


Definition 5 
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2.4.3 Recurrence Relations 


In Examples 1-3 we specified sequences by providing explicit formulas for their terms. There 
are many other ways to specify a sequence. For example, another way to specify a sequence is 
to provide one or more initial terms together with a rule for determining subsequent terms from 
those that precede them. 


A recurrence relation for the sequence {a,} is an equation that expresses a, in terms of 
one or more of the previous terms of the sequence, namely, dp, 44, ..., d,_1, for all integers n 
with n > no, where ng is a nonnegative integer. A sequence is called a solution of a recurrence 
relation if its terms satisfy the recurrence relation. (A recurrence relation is said to recursively 
define a sequence. We will explain this alternative terminology in Chapter 5.) 


Let {a,,} be a sequence that satisfies the recurrence relation a, = a,_, +3 for n = 1, 2,3,..., 
and suppose that ay = 2. What are a, a), and a3? 


Solution: We see from the recurrence relation that a} = dy +3 = 2 +3 = 5. It then follows that 
a =54+3=8anda,=8+3=11. < 


Let {a,,} be a sequence that satisfies the recurrence relation a„ = a,_; — 4,2 fOr n = 2, 3,4,..., 
and suppose that dy = 3 and a, = 5. What are a, and a3? 


Solution: We see from the recurrence relation that a, = a, — dg = 5 —3 = 2 and a; =a, —a, = 
2 — 5 = —3. We can find a4, as, and each successive term in a similar way. < 


The initial conditions for a recursively defined sequence specify the terms that precede the 
first term where the recurrence relation takes effect. For instance, the initial condition in Ex- 
ample 5 is dg = 2, and the initial conditions in Example 6 are ay = 3 and a, = 5. Using mathe- 
matical induction, a proof technique introduced in Chapter 5, it can be shown that a recurrence 
relation together with its initial conditions determines a unique solution. 

Next, we define a particularly useful sequence defined by a recurrence relation, known as 
the Fibonacci sequence, after the Italian mathematician Fibonacci who was born in the 12th 
century (see Chapter 5 for his biography). We will study this sequence in depth in Chapters 
5 and 8, where we will see why it is important for many applications, including modeling the 
population growth of rabbits. Fibonacci numbers occur naturally in the structures of plants and 
animals, such as in the arrangement of sunflower seeds in a seed head and in the shell of the 
chambered nautilus. 


The Fibonacci sequence, fo, fi fo, .-., is defined by the initial conditions fọ = 0, fi = 1, and 
the recurrence relation 


Sn = fa- T n—2 


OER OA 


168 2/ Basic Structures: Sets, Functions, Sequences, Sums, and Matrices 


EXAMPLE 7 


EXAMPLE 8 


EXAMPLE 9 


EXAMPLE 10 


Find the Fibonacci numbers fz, f3, f4» fs, and fe. 


Solution: The recurrence relation for the Fibonacci sequence tells us that we find successive 
terms by adding the previous two terms. Because the initial conditions tell us that fọ = 0 and 
fı = 1, using the recurrence relation in the definition we find that 


h=f th =1+0=1 
h=hth =14+1=2, 
s =h th =24+1=3, 
fs =f th =3+2=5, 
fs =fs thy =5+3=8. < 


Suppose that {a„} is the sequence of integers defined by a, =n!, the value of the facto- 


rial function at the integer n, where n = 1, 2,3, .... Because n! = n((n — 1)(n—-2)...2-1) = 
n(n— 1)! =na,_,, we see that the sequence of factorials satisfies the recurrence relation 
d, = na,_,, together with the initial condition a, = 1. < 


We say that we have solved the recurrence relation together with the initial conditions when 
we find an explicit formula, called a closed formula, for the terms of the sequence. 


Determine whether the sequence {a,,}, where a,, = 3n for every nonnegative integer n, is a solu- 
tion of the recurrence relation a,, = 2a,_, — d,_ for n = 2, 3,4, ... . Answer the same question 
where a, = 2” and where a, = 5. 


Solution: Suppose that a, = 3n for every nonnegative integer n. Then, for n > 2, we see that 
24,1 — 4,2 = 2(3(n — 1)) — 3(n — 2) = 3n = a„. Therefore, {a,,}, where a,, = 3n, is a solution 
of the recurrence relation. 

Suppose that a„ = 2” for every nonnegative integer n. Note that aj = 1,a, = 2, anda, = 4. 
Because 2a, — dy = 2:2—1=3 # a, we see that {a,}, where a, = 2”, is not a solution of the 
recurrence relation. 

Suppose that a, =5 for every nonnegative integer n. Then for n > 2, we see that a, = 
24y-1 — 4-2 = 2: 5-5 =5 = a„. Therefore, {a,,}, where a, = 5, is a solution of the recur- 
rence relation. < 


Many methods have been developed for solving recurrence relations. Here, we will intro- 
duce a straightforward method known as iteration via several examples. In Chapter 8 we will 
study recurrence relations in depth. In that chapter we will show how recurrence relations can 
be used to solve counting problems and we will introduce several powerful methods that can be 
used to solve many different recurrence relations. 


Solve the recurrence relation and initial condition in Example 5. 
Solution: We can successively apply the recurrence relation in Example 5, starting with the 


initial condition a, = 2, and working upward until we reach a, to deduce a closed formula for 
the sequence. We see that 


ay =24+3 
ad, = (2+3)4+3=24+3-2 
dg =(24+2-3)4+3=243-3 


y =a 1 +3 =(24+3-(n—-2)) +3 =243(n— 1). 


EXAMPLE 11 


Extra 
Examples 


» 
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We can also successively apply the recurrence relation in Example 5, starting with the term 
a, and working downward until we reach the initial condition a, = 2 to deduce this same for- 
mula. The steps are 


dy = 4,1 +3 
= (a,-2 +3) +3 =a,.+3:-2 
=(a,_3+3)+3-2=a,3,4+3°-3 


=a, +3(n—-2) = (a, +3) +3 —-2) =24+ 3-1). 


At each iteration of the recurrence relation, we obtain the next term in the sequence by 
adding 3 to the previous term. We obtain the nth term after n — 1 iterations of the recurrence 
relation. Hence, we have added 3(n — 1) to the initial term ay = 2 to obtain a„. This gives us the 
closed formula a, = 2 + 3(n — 1). Note that this sequence is an arithmetic progression. < 


The technique used in Example 10 is called iteration. We have iterated, or repeatedly used, 
the recurrence relation. The first approach is called forward substitution—we found successive 
terms beginning with the initial condition and ending with a,. The second approach is called 
backward substitution, because we began with a, and iterated to express it in terms of falling 
terms of the sequence until we found it in terms of a,. Note that when we use iteration, we 
essentialy guess a formula for the terms of the sequence. To prove that our guess is correct, we 
need to use mathematical induction, a technique we discuss in Chapter 5. 

In Chapter 8 we will show that recurrence relations can be used to model a wide variety of 
problems. We provide one such example here, showing how to use a recurrence relation to find 
compound interest. 


Compound Interest Suppose that a person deposits $10,000 in a savings account at a bank 
yielding 11% per year with interest compounded annually. How much will be in the account 
after 30 years? 


Solution: To solve this problem, let P,, denote the amount in the account after n years. Because 
the amount in the account after n years equals the amount in the account after n — 1 years plus 
interest for the nth year, we see that the sequence {P,,} satisfies the recurrence relation 


P, = P, +0.11P, =(L.1DP,.4. 


The initial condition is Py = 10,000. 
We can use an iterative approach to find a formula for P,,. Note that 


P, =(1.11)Po 
P, =(11DP, S011) 7, 
P, = (1.11)P, = (1.1 1)°P9 
P,=(L1DP,_) = (1.1 1D"Po. 
When we insert the initial condition Py = 10,000, the formula P,, = (1.11)”10,000 is obtained. 
Inserting n = 30 into the formula P,, = (1.11)"10,000 shows that after 30 years the account 


contains 


Py = (1.11)°°10,000 = $228,922.97. < 


170 2/Basic Structures: Sets, Functions, Sequences, Sums, and Matrices 


EXAMPLE 12 


Extra > 
Examples 


EXAMPLE 13 


EXAMPLE 14 


2.4.4 Special Integer Sequences 


A common problem in discrete mathematics is finding a closed formula, a recurrence relation, 
or some other type of general rule for constructing the terms of a sequence. Sometimes only a 
few terms of a sequence solving a problem are known; the goal is to identify the sequence. Even 
though the initial terms of a sequence do not determine the entire sequence (after all, there are 
infinitely many different sequences that start with any finite set of initial terms), knowing the 
first few terms may help you make an educated conjecture about the identity of your sequence. 
Once you have made this conjecture, you can try to verify that you have the correct sequence. 
When trying to deduce a possible formula, recurrence relation, or some other type of rule 
for the terms of a sequence when given the initial terms, try to find a pattern in these terms. You 
might also see whether you can determine how a term might have been produced from those 
preceding it. There are many questions you could ask, but some of the more useful are: 


> Are there runs of the same value? That is, does the same value occur many times in a row? 


> Are terms obtained from previous terms by adding the same amount or an amount that 
depends on the position in the sequence? 


> Are terms obtained from previous terms by multiplying by a particular amount? 
> Are terms obtained by combining previous terms in a certain way? 
> Are there cycles among the terms? 


Find formulae for the sequences with the following first five terms: (a) 1, 1/2, 1/4, 1/8, 1/16 
(b) 1,3,5,7,9 (c)1,-1,1,-1, 1. 


Solution: (a) We recognize that the denominators are powers of 2. The sequence witha, = 1/2", 
n =0, 1,2, ... isa possible match. This proposed sequence is a geometric progression witha = 1 
andr = 1/2. 

(b) We note that each term is obtained by adding 2 to the previous term. The sequence 
with a, = 2n + 1,n=0, 1,2, ... is a possible match. This proposed sequence is an arithmetic 
progression with a = 1 and d = 2. 

(c) The terms alternate between 1 and —1. The sequence with a, = (-1)",n = 0, 1,2... is 
a possible match. This proposed sequence is a geometric progression with a = 1 andr = —1. 


< 


Examples 13-15 illustrate how we can analyze sequences to find how the terms are con- 
structed. 


How can we produce the terms of a sequence if the first 10 terms are 1, 2, 2, 3, 3, 3, 4, 4, 4, 4? 


Solution: In this sequence, the integer 1 appears once, the integer 2 appears twice, the integer 3 
appears three times, and the integer 4 appears four times. A reasonable rule for generating this 
sequence is that the integer n appears exactly n times, so the next five terms of the sequence 
would all be 5, the following six terms would all be 6, and so on. The sequence generated this 
way is a possible match. < 


How can we produce the terms of a sequence if the first 10 terms are 5, 11, 17, 23, 29, 35, 41, 
47, 53, 59? 


Solution: Note that each of the first 10 terms of this sequence after the first is obtained by adding 
6 to the previous term. (We could see this by noticing that the difference between consecutive 
terms is 6.) Consequently, the nth term could be produced by starting with 5 and adding 6 a total 


EXAMPLE 15 


Courtesy of Neil Sloane 
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TABLE 1 Some Useful Sequences. 


First 10 Terms 


1, 4, 9, 16, 25, 36, 49, 64, 81, 100, ... 
1, 8, 27, 64, 125, 216, 343, 512, 729, 1000, ... 


1, 16, 81, 256, 625, 1296, 2401, 4096, 6561, 10000, ... 
1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ... 

2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, ... 

3, 9, 27, 81, 243, 729, 2187, 6561, 19683, 59049, ... 

1, 2, 6, 24, 120, 720, 5040, 40320, 362880, 3628800, ... 


of n — 1 times; that is, a reasonable guess is that the nth term is 5 + 6(n — 1) = 6n — 1. (This is 
an arithmetic progression with a = 5 and d = 6.) < 


How can we produce the terms of a sequence if the first 10 terms are 1, 3, 4, 7, 11, 18, 29, 47, 
76, 123? 


Solution: Observe that each successive term of this sequence, starting with the third term, is 
the sum of the two previous terms. That is, 4 = 3+ 1,7 = 4 +3, 11 = 7 + 4, and so on. Conse- 
quently, if L,, is the nth term of this sequence, we guess that the sequence is determined by the 
recurrence relation L, = L,_; + L,_. with initial conditions L, = 1 and L, = 3 (the same recur- 
rence relation as the Fibonacci sequence, but with different initial conditions). This sequence is 
known as the Lucas sequence, after the French mathematician François Edouard Lucas. Lucas 
studied this sequence and the Fibonacci sequence in the nineteenth century. < 


Another useful technique for finding a rule for generating the terms of a sequence is to 
compare the terms of a sequence of interest with the terms of a well-known integer sequence, 
such as terms of an arithmetic progression, terms of a geometric progression, perfect squares, 
perfect cubes, and so on. The first 10 terms of some sequences you may want to keep in mind are 
displayed in Table 1. Note that we have listed these sequences so that the terms of each sequence 
grow faster than those in the preceding sequence in the list. The rates of growth of these terms 
will be studied in Section 3.2. 


NEIL SLOANE (BORN 1939) Neil Sloane studied mathematics and electrical engineering at the University 
of Melbourne on a scholarship from the Australian state telephone company. He mastered many telephone- 
related jobs, such as erecting telephone poles, in his summer work. After graduating, he designed minimal-cost 
telephone networks in Australia. In 1962 he came to the United States and studied electrical engineering at 
Cornell University. His Ph.D. thesis was on what are now called neural networks. He took a job at Bell Labs 
in 1969, working in many areas, including network design, coding theory, and sphere packing. He moved to 
AT&T Labs in 1996 when it was split off from Bell Labs, working there until his retirement in 2012. One 
of his favorite problems is the kissing problem (a name he coined), which asks how many spheres can be 
arranged in n dimensions so that they all touch a central sphere of the same size. (In two dimensions the an- 
swer is 6, because 6 pennies can be placed so that they touch a central penny. In three dimensions, 12 billiard 


balls can be placed so that they touch a central billiard ball. Two billiard balls that just touch are said to “kiss,” giving rise to the 
terminology “kissing problem” and “kissing number.”) Sloane, together with Andrew Odlyzko, showed that in 8 and 24 dimensions, 
the optimal kissing numbers are, respectively, 240 and 196,560. The kissing number is known in dimensions 1, 2, 3, 4, 8, and 24, but 
not in any other dimensions. Sloane’s books include Sphere Packings, Lattices and Groups, 3d ed., with John Conway; The Theory 
of Error-Correcting Codes with Jessie MacWilliams; The Encyclopedia of Integer Sequences with Simon Plouffe (which has grown 
into the popular OEIS website); and The Rock-Climbing Guide to New Jersey Crags with Paul Nick. The last book demonstrates his 
interest in rock climbing; it includes more than 50 climbing sites in New Jersey. 
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EXAMPLE 16 


Check out the puzzles at 


the OEIS site. 


Links » 


Conjecture a simple formula for a, if the first 10 terms of the sequence {a,} are 1, 7, 25, 79, 
241, 727, 2185, 6559, 19681, 59047. 


Solution: To attack this problem, we begin by looking at the difference of consecutive terms, 
but we do not see a pattern. When we form the ratio of consecutive terms to see whether each 
term is a multiple of the previous term, we find that this ratio, although not a constant, is close 
to 3. So it is reasonable to suspect that the terms of this sequence are generated by a formula 
involving 3”. Comparing these terms with the corresponding terms of the sequence {3”}, we 
notice that the nth term is 2 less than the corresponding power of 3. We see that a, = 3” — 2 for 
1 <n < 10 and conjecture that this formula holds for all n. < 


We will see throughout this text that integer sequences appear in a wide range of contexts in 
discrete mathematics. Sequences we have encountered or will encounter include the sequence 
of prime numbers (Chapter 4), the number of ways to order n discrete objects (Chapter 6), the 
number of moves required to solve the Tower of Hanoi puzzle with n disks (Chapter 8), and the 
number of rabbits on an island after n months (Chapter 8). 

Integer sequences appear in an amazingly wide range of subject areas besides discrete math- 
ematics, including biology, engineering, chemistry, and physics, as well as in puzzles. An amaz- 
ing database of over 250,000 different integer sequences (as of 2017) can be found in the On- 
Line Encyclopedia of Integer Sequences (OEIS). This database was originated by Neil Sloane in 
1964 and is now maintained by the OEIS Foundation. The last printed version of this database 
was published in 1995 ({SIPI95]); the current encyclopedia would occupy more than 900 vol- 
umes of the size of the 1995 book with more than 10,000 new submissions a year. You can use a 
program on the OEIS website to find sequences from the encyclopedia that match initial terms 
you provide, if there is a match. For instance, when you enter 1, 1, 2, 3, 5, 8, OEIS displays a 
page that identifies these numbers as successive terms of the Fibonacci sequence, provides the 
recurrence relation that generates this sequence, lists an extensive set of comments about the 
many ways the Fibonacci sequence arises including references, and displays information about 
quite a few other sequences that begin with these same terms. 


2.4.5 Summations 


Next, we consider the addition of the terms of a sequence. For this we introduce summation 
notation. We begin by describing the notation used to express the sum of the terms 


Any m+ +++ An 


from the sequence {a,,}. We use the notation 


n 
5 đj, È- mij? or marén a; 


(read as the sum from j = m to j = n of a;) to represent 
am T Am+1 i an: 


Here, the variable j is called the index of summation, and the choice of the letter j as the variable 
is arbitrary; that is, we could have used any other letter, such as i or k. Or, in notation, 


EXAMPLE 17 


Extra » 
Examples 


EXAMPLE 18 


EXAMPLE 19 


EXAMPLE 20 
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Here, the index of summation runs through all integers starting with its lower limit m and ending 
with its upper limit n. A large uppercase Greek letter sigma, >’, is used to denote summation. 

The usual laws for arithmetic apply to summations. For example, when a and b are real 
numbers, we have };._, (ax; + by;) = a X1% + b Ye, Yj Where x1, Xp, ..-,%, ANA Yi, Yo -+5 Yp 
are real numbers. (We do not present a formal proof of this identity here. Such a proof can 
be constructed using mathematical induction, a proof method we introduce in Chapter 5. The 
proof also uses the commutative and associative laws for addition and the distributive law of 
multiplication over addition.) 

We give some examples of summation notation. 


Use summation notation to express the sum of the first 100 terms of the sequence {a;}, where 
a; = 1/j forj = 1,2,3,.... 


Solution: The lower limit for the index of summation is 1, and the upper limit is 100. We write 
this sum as 


What is the value of a J? 


Solution: We have 


5 
MP HSV tVs 2+ e4s? 


j=l 
=1+4+9+4 16425 
= 55. < 
What is the value of ye _ (1)? 
Solution: We have 
8 
LOD = bt + CDS + OD + C7 + (ED 
k=4 
=14+(-1I+1+(-1)+1 
< 


=Í. 


Sometimes it is useful to shift the index of summation in a sum. This is often done when 
two sums need to be added but their indices of summation do not match. When shifting an index 
of summation, it is important to make the appropriate changes in the corresponding summand. 
This is illustrated by Example 20. 


Suppose we have the sum 


174 2/Basic Structures: Sets, Functions, Sequences, Sums, and Matrices 


Extra » but want the index of summation to run between 0 and 4 rather than from 1 to 5. To do this, we 
Examples let k = j — 1. Then the new summation index runs from 0 (because k = 1 — 0 = 0 when j = 1) 
to 4 (because k = 5 — 1 = 4 when j = 5), and the term j? becomes (k + 1)”. Hence, 


5 4 
Xr = a+ 1). 
j=l k=0 
It is easily checked that both sums are 1 +4 +9 + 16 +25 = 55. < 


Sums of terms of geometric progressions commonly arise (such sums are called geometric 
series). Theorem 1 gives us a formula for the sum of terms of a geometric progression. 


THEOREM 1 If a and r are real numbers and r # 0, then 


ives Il 


j=0 (n+ la ifr=l. 


Proof: Let 
S, = X ar. 
j=0 


To compute S, first multiply both sides of the equality by r and then manipulate the resulting 
sum as follows: 


n 


rs, =F > ar! substituting summation formula for S 
j=0 
n 
= £ ar'+! by the distributive property 
j=0 
n+l 
= > ar’ shifting the index of summation, with k = j+ 1 
k=1 
n 
= 5 art | +(ar"t! —a) removing k =n + 1 term and adding k = 0 term 
k=0 
= S, + (ar"*! — a) substituting S for summation formula 


From these equalities, we see that 
rS, = S, + (ar"*! — a). 
Solving for S„ shows that if r 4 1, then 
ar"! —a 
~ r-l 


If r = 1, then the S, = X04 = Ya = (n + La. < 


EXAMPLE 21 


EXAMPLE 22 


EXAMPLE 23 
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Double summations arise in many contexts (as in the analysis of nested loops in computer pro- 
grams). An example of a double summation is 


To evaluate the double sum, first expand the inner summation and then continue by computing 
the outer summation: 


4 3 4 
> YG + 2i +31) 


=64+124+184+24=60. < 


We can also use summation notation to add all values of a function, or terms of an indexed 
set, where the index of summation runs over all values in a set. That is, we write 


YO 


ses 
to represent the sum of the values f(s), for all members s of S. 


What is the value of }). 10,24} 5? 


Solution: Because J, e ¢9,,4; 5 represents the sum of the values of s for all the members of the 
set {0, 2, 4}, it follows Shut 


s=0+2+4=6. 
se {0,2,4} < 


Certain sums arise repeatedly throughout discrete mathematics. Having a collection of for- 
mulae for such sums can be useful; Table 2 provides a small table of formulae for commonly 
occurring sums. 

We derived the first formula in this table in Theorem 1. The next three formulae give us the 
sum of the first n positive integers, the sum of their squares, and the sum of their cubes. These 
three formulae can be derived in many different ways (for example, see Exercises 37 and 38). 
Also note that each of these formulae, once known, can be proved using mathematical induction, 
the subject of Section 5.1. The last two formulae in the table involve infinite series and will be 
discussed shortly. 

Example 23 illustrates how the formulae in Table 2 can be useful. 


Find Ye, 


Solution: First note that because Dor ? k + pe a we have 


100 100 


2D ye 


k=50 
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TABLE 2 Some Useful Summation Formulae. 


Closed Form 


+l O 
a TA rH 
r-1 
n(n + 1) 
2 


n(n + 1)(2n + 1) 
6 


n(n +1) 


Using the formula De A = n(n + 1)(2n + 1)/6 from Table 2 (and proved in Exercise 38), we 


see that 
= 100-101-201 49-50-99 
5, K= ea L Z = 338,350 — 40,425 = 297,925. 
k=50 6 6 < 


SOME INFINITE SERIES Although most of the summations in this book are finite sums, 
infinite series are important in some parts of discrete mathematics. Infinite series are usually 
studied in a course in calculus and even the definition of these series requires the use of calcu- 
lus, but sometimes they arise in discrete mathematics, because discrete mathematics deals with 
infinite collections of discrete elements. In particular, in our future studies in discrete mathe- 
matics, we will find the closed forms for the infinite series in Examples 24 and 25 to be quite 
useful. 


EXAMPLE 24 (Requires calculus) Let x be a real number with |x| < 1. Find Do aie 


k+l _ 
Gta Solution: By Theorem 1 witha = 1 and r = x we see that Lor =* l . Because |x| < 1, 
Examples » x+! approaches 0 as k approaches infinity. It follows that 
o9 k+l _ 2 
v= lim Ë 1_0-1_ 1 
= k>% x—1 x-1l 1-x <q 


We can produce new summation formulae by differentiating or integrating existing formulae. 


EXAMPLE 25 (Requires calculus) Differentiating both sides of the equation 


`A 1 
ar 1-x’ 


from Example 24 we find that 


Dee “a Tr 
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(This differentiation is valid for |x| < 1 by a theorem about infinite series.) < 


Exercises 


1. Find these terms of the sequence {a,}, where a, = 


2-(—3)" +5". 
a) ap b) a, c) a, d) a; 
. What is the term a, of the sequence {a,,} if a, equals 
a) 27-19 b) 7? 
c) 1+(-1)”? d) —(—2)"? 


. What are the terms ag, a), ay, and a; of the sequence {a,,}, 
where a,, equals 

a) 2” +1? b) (n+ 1)+!? 

c) [n/2]? d) |n/2] + [n/2]? 

. What are the terms dp, a, dy, and a; of the sequence {a,,}, 
where a,, equals 

a) (—2)"? b) 3? 

c) 7+4"? d) 2” + (—2)"? 

. List the first 10 terms of each of these sequences. 


a) the sequence that begins with 2 and in which each 
successive term is 3 more than the preceding term 

b) the sequence that lists each positive integer three 
times, in increasing order 

c) the sequence that lists the odd positive integers in in- 
creasing order, listing each odd integer twice 

d) the sequence whose nth term is n! — 2” 

e) the sequence that begins with 3, where each succeed- 
ing term is twice the preceding term 

f) the sequence whose first term is 2, second term is 4, 
and each succeeding term is the sum of the two pre- 
ceding terms 

g) the sequence whose nth term is the number of bits 
in the binary expansion of the number n (defined in 
Section 4.2) 

h) the sequence where the nth term is the number of let- 
ters in the English word for the index n 

. List the first 10 terms of each of these sequences. 

a) the sequence obtained by starting with 10 and obtain- 
ing each term by subtracting 3 from the previous term 

b) the sequence whose nth term is the sum of the first n 
positive integers 

c) the sequence whose nth term is 3” — 2” 

d) the sequence whose nth term is Lyn] 

e) the sequence whose first two terms are 1 and 5 and 
each succeeding term is the sum of the two previous 
terms 

f) the sequence whose nth term is the largest integer 
whose binary expansion (defined in Section 4.2) has 
n bits (Write your answer in decimal notation.) 


10. 


11. 


12. 


13. 


14. 


g) the sequence whose terms are constructed sequen- 
tially as follows: start with 1, then add 1, then mul- 
tiply by 1, then add 2, then multiply by 2, and so on 

h) the sequence whose nth term is the largest integer k 
such that k! < n 


. Find at least three different sequences beginning with the 


terms 1, 2, 4 whose terms are generated by a simple for- 
mula or rule. 


. Find at least three different sequences beginning with the 


terms 3, 5, 7 whose terms are generated by a simple for- 
mula or rule. 


. Find the first five terms of the sequence defined by each 


of these recurrence relations and initial conditions. 


a) a, = 6a,_), dy =2 
a 
b) a, =a 4 =2 
c) a, =4,_; +34,2; 4) = 1,a, =2 


d) a, = na, +na, 2 a = la, =1 

e) a, = 4,1 +4,3, Ay = l,a; = 2, a, =0 

Find the first six terms of the sequence defined by each 
of these recurrence relations and initial conditions. 


a) a, = —2d,_),d4) = —1 

b) a, =4,_, — 4,2, 4g = 2,a, = —1 
c) a, = 3a, ay = 1 

d) a, = na, +a, dy = —l,a, =0 


e) a, = 4,1 — 4,2 +4,3 4o = Lay = Lay =2 
Let a, = 2” +5- 3” forn =0,1,2,.... 
a) Find dp, dj, dy, a3, and ay. 


b) Show that a, = 5a, — 6a, a; = Say — 6a,, and a, = 
5a, — 6a}. 

c) Show that a, = Sa,_; — 6a,_» for all integers n with 
n>2. 


Show that the sequence {a,,} is a solution of the recur- 
rence relation a, = —3a,_, + 4a,_, if 


a) a, =0. b) a, =1. 

c) a, = (-4)”. d) a, = 2(-4)" +3. 

Is the sequence {a,,} a solution of the recurrence relation 
a, = 8a,_, — 16a,_, if 

a) a, =0? b) a, =1? 

c) a, = 2"? d) a, =4"? 

e) a, = n4"? f) a, =2-4"+3n4"? 

g) a, = (—4)"? h) a, = n°4"? 


For each of these sequences find a recurrence relation 
satisfied by this sequence. (The answers are not unique 
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15. 


16. 


17. 


18. 


19. 


20. 


Links > 


because there are infinitely many different recurrence 
relations satisfied by any sequence.) 


a) a, =3 b) a, =2n 
c) a, =2n+3 d) a, = 5” 
e) a, =n f) ap=n°+n 
g) a,=n+(-l)" h) a, =n! 


Show that the sequence {a,,} is a solution of the recur- 
rence relation a, = d,_; + 2a,_, + 2n — 9 if 

a) a,=—n+2. 

b) a, = 5(-1) -n +2. 

c) a, =3(—1) +2" -n +2. 

d) a, =7-2"-—n+2. 

Find the solution to each of these recurrence relations 
with the given initial conditions. Use an iterative ap- 
proach such as that used in Example 10. 


a) a, = —a,_),d49 =5 

b) a, =4,_, +3,49 =1 

c) ad, =4,_; —N, da) =4 

d) a, = 2a,_, — 3, a& = —1 

e) a, = (n + 1)a,_),d9 = 2 
f) a, = 2na, 49 = 3 

g) a, =—-a,_,+n-1,a)=7 


Find the solution to each of these recurrence relations and 
initial conditions. Use an iterative approach such as that 
used in Example 10. 

a) a, = 3a,_),d) = 2 

b) a, =4,_; +2,49 =3 


c) a, = a, +A, ag = 1 
d) a, =4,_, + 2n +3, ag = 4 
e) a, =2a,_,; —1,a) = 1 


f) a, =3a,_;+1,a4)=1 
g) a, =Nna,_,,4) =5 

h) a, = 2na 
A person deposits $1000 in an account that yields 9% in- 
terest compounded annually. 


n-1> 49 = 1 


a) Setup a recurrence relation for the amount in the ac- 
count at the end of n years. 

b) Find an explicit formula for the amount in the account 
at the end of n years. 

c) How much money will the account contain after 100 
years? 

Suppose that the number of bacteria in a colony triples 

every hour. 

a) Setup a recurrence relation for the number of bacteria 
after n hours have elapsed. 

b) If 100 bacteria are used to begin a new colony, how 
many bacteria will be in the colony in 10 hours? 
Assume that the population of the world in 2017 was 7.6 

billion and is growing at the rate of 1.12% a year. 

a) Set up a recurrence relation for the population of the 
world n years after 2017. 

b) Find an explicit formula for the population of the 
world n years after 2017. 

c) What will the population of the world be in 2050? 


21. 


22. 


23. 


24. 


Links » 


25. 


26. 


6? 27: 


A factory makes custom sports cars at an increasing rate. 
In the first month only one car is made, in the second 
month two cars are made, and so on, with n cars made in 
the nth month. 


a) Set up a recurrence relation for the number of cars 
produced in the first n months by this factory. 

b) How many cars are produced in the first year? 

c) Find an explicit formula for the number of cars pro- 
duced in the first n months by this factory. 


An employee joined a company in 2017 with a starting 
salary of $50,000. Every year this employee receives a 
raise of $1000 plus 5% of the salary of the previous year. 


a) Set up a recurrence relation for the salary of this em- 
ployee n years after 2017. 

b) What will the salary of this employee be in 2025? 

c) Find an explicit formula for the salary of this em- 
ployee n years after 2017. 


Find a recurrence relation for the balance B(k) owed at 
the end of k months on a loan of $5000 at a rate of 7% 
if a payment of $100 is made each month. [Hint: Ex- 
press B(k) in terms of B(k — 1); the monthly interest is 
(0.07/12) B(k — 1).] 
a) Find arecurrence relation for the balance B(k) owed at 
the end of k months on a loan ata rate of r if a payment 
P is made on the loan each month. [Hint: Express 
B(k) in terms of B(k — 1) and note that the monthly 
interest rate is r/12.] 
b) Determine what the monthly payment P should be so 
that the loan is paid off after T months. 

For each of these lists of integers, provide a simple for- 
mula or rule that generates the terms of an integer se- 
quence that begins with the given list. Assuming that your 
formula or rule is correct, determine the next three terms 
of the sequence. 
a) 1,0, 1,1,0,0, 1,1,1, 
b) 1, 
c) 1, 
d) 3, 
e) 15, ae —13, —20, —27, . 
f) 3, 5,8, 12, 17, 23, 30, 38, 47, . 

g) 2, 16,54, 128, 250, 432, 686, . 
h) 2, 3, 7, 25, 121, 721, 5041, 40321, . 
For each of these lists of integers, pii a simple for- 
mula or rule that generates the terms of an integer se- 
quence that begins with the given list. Assuming that your 
formula or rule is correct, determine the next three terms 
of the sequence. 
a) 3,6, 11, 18, 27, 38, 51, 66, 83, 102, ... 
b) 7, 11, 15, 19, 23, 27, 31, 35, 39, 43, ... 


c) 1, 10, 11, 100, 101, 110, 111, 1000, 1001, 1010, 1011, ... 
d) 1,22, 2,3; 3; 3; 3,355.55): 9) 395 Dp aa 

e) 0, 2, 8, 26, 80, 242, 728, 2186, 6560, 19682, ... 

f) 1,3, 15, 105, 945, 10395, 135135, 2027025, 34459425, ... 
g)100L110000LLL1L1... 


h) 2,4, 16, 256, 65536, 4294967296, ... 
Show that if a„ denotes the nth positive integer that is not 


a perfect square, thena, =n + { yn }, where {x} denotes 
the integer closest to the real number x. 


* 28. 


29. 


30. 


31. 


32. 


33. 


34. 


Let a, be the nth term of the sequence 1, 2, 2, 3, 3, 3, 
4, 4, 4, 4,5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6,..., constructed 
by including the integer k exactly k times. Show that 


an = LV2n+ iJ. 


What are the values of these sums? 
5 4 
a) Ð (k+1) b) $, (-2y 
k=1 j=0 
10 8 
0 > 3 da) $ (2+! -2/) 
i=l j=0 
What are the values of these sums, where S = 
{1, 3,5, 7}? 
a) Èj b) YF 
JES jes 
0 Yds) d yl 
JES jes 
What is the value of each of these sums of terms of a 
geometric progression? 
8 8 
a) © 3-2) ) È 
j=0 j=l 
c) > (-3y d) > 2-(-3y 
j=2 j=0 
Find the value of each of these sums. 
8 8 
a) D(1+(-1)) b) Li-2) 
j=0 j=0 
8 
o0 Y(2-34+3-2/) da) $ (2+! -2/) 
j=0 j=0 
Compute each of these double sums. 
2 3 2 3 
a) Y X(+) b) X }2i+3j) 
i= <r 1 i=0j=0 
c) 5 A i d È da 
i=1j= i=0j=1 


Compute cia of these double sums. 


3 2 ag 
a) È X- b) Y YGi+t2/) 


ers a 


c) Ey DIE 


j i=0j= 


Cardinality of Sets 


2.5.1 Introduction 
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35. Show that $; (4-41) =a,— ap where 
dy, 4, ..., 4, is a sequence of real numbers. This type 
of sum is called telescoping. 

36. Use the identity 1/(k(k + 1)) = 1/k — 1/(k + 1) and Ex- 
ercise 35 to compute YY" _, 1/(k(k + 1)). 
37. Sum both sides of the identity k? — (k — 1} = 
from k = 1 to k = n and use Exercise 35 to find 
a) a formula for 2 (2k — 1) (the sum of the first n 
odd natural numbers). 
b) a formula for }))_ | k. 

* 38. Use the technique given in Exercise 35, together with the 
result of Exercise 37b, to derive the formula for >= i k? 
given in Table 2. [Hint: Take a, = k? in the telescoping 
sum in P e 35.] 


39. Find ye 100K: (Use Table 2.) 

40. Find 3°°..k°. (Use Table 2.) 

41. Find 7 K(k — 3). (Use Table 2.) 

42. Find pa 19k — 1)(2? + 1). (Use Table 2.) 


* 43, Find a formula for Evel, when m is a positive 


2k —1 


integer. 
* 44. Find a formula for )7"_, | /k], when m is a positive in- 
teger. 


There is also a special notation for products. The product of 


Any m+ +++ 


a, is represented by |] 


jam 


aj, read as the product 


from j = m to j =n of aj. 
45. What are the values of the following products? 
a) Itoi j b) Iisi 
100 10 
© JCD d) J[;;2 
Recall that the value of the factorial function at a positive in- 


teger n, denoted by n!, is the product of the positive integers 
from 1 to n, inclusive. Also, we specify that 0! = 1. 


46. Express n! using product notation. 
47. Find Yj! 

: 4 
48. Find Il-0)! 


In Definition 4 of Section 2.1 we defined the cardinality of a finite set as the number of elements 
in the set. We use the cardinalities of finite sets to tell us when they have the same size, or when 
one is bigger than the other. In this section we extend this notion to infinite sets. That is, we will 
define what it means for two infinite sets to have the same cardinality, providing us with a way 
to measure the relative sizes of infinite sets. 

We will be particularly interested in countably infinite sets, which are sets with the same 
cardinality as the set of positive integers. We will establish the surprising result that the set of 
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Definition 1 


Definition 2 


Definition 3 


EXAMPLE 1 


rational numbers is countably infinite. We will also provide an example of an uncountable set 
when we show that the set of real numbers is not countable. 

The concepts developed in this section have important applications to computer science. A 
function is called uncomputable if no computer program can be written to find all its values, 
even with unlimited time and memory. We will use the concepts in this section to explain why 
uncomputable functions exist. 

We now define what it means for two sets to have the same size, or cardinality. In Section 2.1, 
we discussed the cardinality of finite sets and we defined the size, or cardinality, of such sets. In 
Exercise 81 of Section 2.3 we showed that there is a one-to-one correspondence between any 
two finite sets with the same number of elements. We use this observation to extend the concept 
of cardinality to all sets, both finite and infinite. 


The sets A and B have the same cardinality if and only if there is a one-to-one correspondence 
from A to B. When A and B have the same cardinality, we write |A| = |B]. 


For infinite sets the definition of cardinality provides a relative measure of the sizes of two sets, 
rather than a measure of the size of one particular set. We can also define what it means for one 
set to have a smaller cardinality than another set. 


If there is a one-to-one function from A to B, the cardinality of A is less than or the same 
as the cardinality of B and we write |A| < |B|. Moreover, when |A| < |B| and A and B have 
different cardinality, we say that the cardinality of A is less than the cardinality of B and we 
write |A| < |B]. 


Remark: In Definitions 1 and 2 we introduced the notations |A| = |B] and |A| < |B| to denote 
that A and B have the same cardinality and that the cardinality of A is less than the cardinality 
of B. However, these definitions do not give any separate meaning to |A| and |B| when A and B 
are arbitrary infinite sets. 


2.5.2 Countable Sets 


We will now split infinite sets into two groups, those with the same cardinality as the set of 
natural numbers and those with a different cardinality. 


A set that is either finite or has the same cardinality as the set of positive integers is called 
countable. A set that is not countable is called uncountable. When an infinite set S is count- 
able, we denote the cardinality of S by Nọ (where N is aleph, the first letter of the Hebrew 
alphabet). We write |S] = Nọ and say that S has cardinality “aleph null.” 


We illustrate how to show a set is countable in the next example. 
Show that the set of odd positive integers is a countable set. 


Solution: To show that the set of odd positive integers is countable, we will exhibit a one-to-one 
correspondence between this set and the set of positive integers. Consider the function 


f(r) =2n-1 


You can always get a 
room at Hilbert’s Grand 


Hotel! 
Links > 


EXAMPLE 2 


Links » 
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PETER 


FIGURE 1 A One-to-One Correspondence Between Z* and the Set of Odd Positive 
Integers. 


from Z* to the set of odd positive integers. We show that f is a one-to-one correspondence by 
showing that it is both one-to-one and onto. To see that it is one-to-one, suppose that f (n) = f(m). 
Then 2n — 1 = 2m — 1, so n = m. To see that it is onto, suppose that ¢ is an odd positive integer. 
Then ¢ is 1 less than an even integer 2k, where k is a natural number. Hence t = 2k — 1 = f(k). 
We display this one-to-one correspondence in Figure 1. < 


An infinite set is countable if and only if it is possible to list the elements of the set in a 
sequence (indexed by the positive integers). The reason for this is that a one-to-one correspon- 
dence f from the set of positive integers to a set S can be expressed in terms of a sequence 
4i, az, ...,4,,..., Where a, = f(1), a, = f (2), ..., a, = f(n), .... 


HILBERT’S GRAND HOTEL We now describe a paradox that shows that something impos- 
sible with finite sets may be possible with infinite sets. The famous mathematician David Hilbert 
invented the notion of the Grand Hotel, which has a countably infinite number of rooms, each 
occupied by a guest. When a new guest arrives at a hotel with a finite number of rooms, and 
all rooms are occupied, this guest cannot be accommodated without evicting a current guest. 
However, we can always accommodate a new guest at the Grand Hotel, even when all rooms are 
already occupied, as we show in Example 2. Exercises 5 and 8 ask you to show that we can ac- 
commodate a finite number of new guests and a countable number of new guests, respectively, 
at the fully occupied Grand Hotel. 


How can we accommodate a new guest arriving at the fully occupied Grand Hotel without 
removing any of the current guests? 


Solution: Because the rooms of the Grand Hotel are countable, we can list them as Room 1, 
Room 2, Room 3, and so on. When a new guest arrives, we move the guest in Room 1 to Room 
2, the guest in Room 2 to Room 3, and in general, the guest in Room n to Room n + 1, for all 
positive integers n. This frees up Room 1, which we assign to the new guest, and all the current 
guests still have rooms. We illustrate this situation in Figure 2. < 


When there are finitely many rooms in a hotel, the notion that all rooms are occupied is 
equivalent to the notion that no new guests can be accommodated. However, Hilbert’s paradox 


DAVID HILBERT (1862-1943) Hilbert, born in Königsberg, the city famous in mathematics for its seven 
bridges, was the son of a judge. During his tenure at Göttingen University, from 1892 to 1930, he made many 
fundamental contributions to a wide range of mathematical subjects. He almost always worked on one area 
of mathematics at a time, making important contributions, then moving to a new mathematical subject. Some 
areas in which Hilbert worked are the calculus of variations, geometry, algebra, number theory, logic, and math- 
ematical physics. Besides his many outstanding original contributions, Hilbert is remembered for his important 
and influential list of 23 unsolved problems. He described these problems at the 1900 International Congress 
of Mathematicians, as a challenge to mathematicians at the birth of the twentieth century. Since that time, they 
have spurred a tremendous amount and variety of research. Although many of these problems have now been 
solved, several remain open, including the Riemann hypothesis, which is part of Problem 8 on Hilbert’s list. 
Hilbert was also the author of several important textbooks in number theory and geometry. 


182 2/ Basic Structures: Sets, Functions, Sequences, Sums, and Matrices 


EXAMPLE 3 
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Take room 1, 
everyone else 
move down one room 


f 


Manager New guest 


FIGURE 2 A new guest arrives at Hilbert’s Grand Hotel. 


of the Grand Hotel can be explained by noting that this equivalence no longer holds when there 
are infinitely many rooms. 


EXAMPLES OF COUNTABLE AND UNCOUNTABLE SETS We will now show that cer- 
tain sets of numbers are countable. We begin with the set of all integers. Note that we can show 
that the set of all integers is countable by listing its members. 


Show that the set of all integers is countable. 


Solution: We can list all integers in a sequence by starting with O and alternating between pos- 
itive and negative integers: 0, 1, —1, 2, —2, .... Alternatively, we could find a one-to-one cor- 
respondence between the set of positive integers and the set of all integers. We leave it to the 
reader to show that the function f(n) = n/2 when n is even and f(n) = —(n — 1)/2 when n is 
odd is such a function. Consequently, the set of all integers is countable. < 


It is not surprising that the set of odd integers and the set of all integers are both countable 
sets (as shown in Examples 1 and 3). Many people are amazed to learn that the set of rational 
numbers is countable, as Example 4 demonstrates. 


Show that the set of positive rational numbers is countable. 


Solution: It may seem surprising that the set of positive rational numbers is countable, 
but we will show how we can list the positive rational numbers as a sequence r}, rz ..., 
r» +... First, note that every positive rational number is the quotient p/q of two positive 
integers. 

We can arrange the positive rational numbers by listing those with denominator g = 1 
in the first row, those with denominator g = 2 in the second row, and so on, as displayed in 
Figure 3. 

The key to listing the rational numbers in a sequence is to first list the positive rational 
numbers p/q with p + q = 2, followed by those with p + q = 3, followed by those with p + q = 
4, and so on, following the path shown in Figure 3. Whenever we encounter a number p/q that 
is already listed, we do not list it again. For example, when we come to 2/2 = 1 we do not 
list it because we have already listed 1/1 = 1. The initial terms in the list of positive rational 
numbers we have constructed are 1, 1/2, 2, 3, 1/3, 1/4, 2/3, 3/2, 4, 5, and so on. These numbers 
are shown circled; the uncircled numbers in the list are those we leave out because they are 


Not all infinite sets have 


the same size! 
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Terms not circled C 
are not listed 


because they 


repeat previously 
listed terms C 6 


3 5 
3 3 
P 4 5 
4 4 4 4 
2 3 4 5 
5 5 5 


FIGURE 3 The positive rational numbers are countable. 


already listed. Because all positive rational numbers are listed once, as the reader can verify, we 
have shown that the set of positive rational numbers is countable. < 


2.5.3 An Uncountable Set 


We have seen that the set of positive rational numbers is a countable set. Do we have a promising 
candidate for an uncountable set? The first place we might look is the set of real numbers. In 
Example 5 we use an important proof method, introduced in 1879 by Georg Cantor and known 
as the Cantor diagonalization argument, to prove that the set of real numbers is not countable. 
This proof method is used extensively in mathematical logic and in the theory of computation. 


Show that the set of real numbers is an uncountable set. 


Solution: To show that the set of real numbers is uncountable, we suppose that the set of real 
numbers is countable and arrive at a contradiction. Then, the subset of all real numbers that 
fall between O and 1 would also be countable (because any subset of a countable set is also 
countable; see Exercise 16). Under this assumption, the real numbers between 0 and 1 can be 
listed in some order, say, r4, r2, r3, .... Let the decimal representation of these real numbers be 


r, = 0.d ddid --- 
n = 0d ddd «.. 
r, = 0.dz dzzd33d34 ... 
r4 = O.dy ddyd ... 


where d; € {0, 1, 2, 3, 4, 5, 6, 7, 8,9}. (For example, if r, = 0.23794102 ..., we have d,, = 
2, dp =3, di3 =7, and so on.) Then, form a new real number with decimal expansion 
r = 0.d,d,d,d, ..., where the decimal digits are determined by the following rule: 


_f4ifdi #4 
|S ifd; = 4. 

(As an example, suppose that r; = 0.23794102..., r, = 0.44590138 ..., r3 = 0.09118764 ..., 
r4 = 0.80553900..., and so on. Then we have r = 0.d,d,d,d, ... = 0.4544 ..., where d} = 4 
because d; # 4, d) = 5 because d, = 4, d} = 4 because dz, # 4, d, = 4 because d,, # 4, and 
so on.) 
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decimal expansion that 
terminates has a second 
decimal expansion 
ending with an infinite 
sequence of 9s because 
1=0.999 .... 


THEOREM 1 


This proof uses WLOG 
and cases. 


THEOREM 2 


Every real number has a unique decimal expansion (when the possibility that the expansion 
has a tail end that consists entirely of the digit 9 is excluded). Therefore, the real number r is not 
equal to any of r}, 7, ... because the decimal expansion of r differs from the decimal expansion 
of r; in the ith place to the right of the decimal point, for each i. 

Because there is a real number r between 0 and 1 that is not in the list, the assumption that all 
the real numbers between 0 and 1 could be listed must be false. Therefore, all the real numbers 
between 0 and 1 cannot be listed, so the set of real numbers between 0 and 1 is uncountable. Any 
set with an uncountable subset is uncountable (see Exercise 15). Hence, the set of real numbers 
is uncountable. < 


RESULTS ABOUT CARDINALITY We will now discuss some results about the cardinality 
of sets. First, we will prove that the union of two countable sets is also countable. 


If A and B are countable sets, then A U B is also countable. 


Proof: Suppose that A and B are both countable sets. Without loss of generality, we can assume 
that A and B are disjoint. (If they are not, we can replace B by B — A, because AN (B— A) = Ø 
and A U (B — A) = A U B.) Furthermore, without loss of generality, if one of the two sets is 
countably infinite and other finite, we can assume that B is the one that is finite. 

There are three cases to consider: (i) A and B are both finite, (ii) A is infinite and B is finite, 
and (iii) A and B are both countably infinite. 


Case (i): Note that when A and B are finite, A U B is also finite, and therefore, countable. 


Case (ii): Because A is countably infinite, its elements can be listed in an infinite sequence 
4i, Ay, d3, ..., Ap, ... and because B is finite, its terms can be listed as b,, bo, ..., b,, for 
some positive integer m. We can list the elements of A U B as by, bo, ..., Dy, Gy, A2, 43, -. 
d,, -... This means that A U B is countably infinite. 


+5 


Case (iii): Because both A and B are countably infinite, we can list their elements as a4, 
An, 43, ..., Ap, ... and by, bo, b3, ..., bpo ..., respectively. By alternating terms of these two 
sequences we can list the elements of A U B in the infinite sequence a,, b,, a2, by, a3, b3, 
-> dy, Das .... This means A U B must be countably infinite. 


n? 


We have completed the proof, as we have shown that A U B is countable in all three 
cases. < 


Because of its importance, we now state a key theorem in the study of cardinality. 


SCHRODER-BERNSTEIN THEOREM IfA and B are sets with |A| < |B| and |B| < 
|A|, then |A| = |B]. In other words, if there are one-to-one functions f from A to B and g from 
B to A, then there is a one-to-one correspondence between A and B. 


Because Theorem 2 seems to be quite straightforward, we might expect that it has an easy 
proof. However, this is not the case because when you have an injection from a set A to a set 
B that may not be onto, and another injection from B to A that also may not be onto, there 
is no obvious way to construct a bijection from A to B. Moreover, even though it has been 
proved in a variety of ways without using advanced mathematics, all known proofs are rather 
subtle and have twists and turns that are tricky to explain. One of these proofs is developed 


EXAMPLE 6 


Definition 4 
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Fraktur c. 
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in Exercise 41 and students are invited to complete the details. Motivated readers can also 
find proofs in [AiZiH009] and [Ve06]. This result is called the Schrdder-Bernstein theorem 
after Ernst Schröder who published a flawed proof of it in 1898 and Felix Bernstein, a stu- 
dent of Georg Cantor, who presented a proof in 1897. However, a proof of this theorem was 
found in notes of Richard Dedekind dated 1887. Dedekind was a German mathematician who 
made important contributions to the foundations of mathematics, abstract algebra, and number 
theory. 
We illustrate the use of Theorem 2 with an example. 


Show that the |(0, 1)| = |(0, 1]]. 


Solution: It is not at all obvious how to find a one-to-one correspondence between (0, 1) and 
(0, 1] to show that |(0, 1)| = |(0, 1]|. Fortunately, we can use the Schréder-Bernstein theorem 
instead. Finding a one-to-one function from (0, 1) to (0, 1] is simple. Because (0, 1) c (0, 1], 
f(x) =x is a one-to-one function from (0, 1) to (0, 1]. Finding a one-to-one function from (0, 1] 
to (0, 1) is also not difficult. The function g(x) = x/2 is clearly one-to-one and maps (0, 1] to 
(0, 1/2] c (0, 1). As we have found one-to-one functions from (0, 1) to (0, 1] and from (0, 1] to 
(0, 1), the Schréder-Bernstein theorem tells us that |(0, 1)| = |(O, 1]. < 


UNCOMPUTABLE FUNCTIONS We will now describe an important application of the con- 
cepts of this section to computer science. In particular, we will show that there are functions 
whose values cannot be computed by any computer program. 


We say that a function is computable if there is a computer program in some programming 
language that finds the values of this function. If a function is not computable we say it is 
uncomputable. 


To show that there are uncomputable functions, we need to establish two results. First, we 
need to show that the set of all computer programs in any particular programming language is 
countable. This can be proved by noting that a computer program in a particular language can 
be thought of as a string of characters from a finite alphabet (see Exercise 37). Next, we show 
that there are uncountably many different functions from a particular countably infinite set to 
itself. In particular, Exercise 38 shows that the set of functions from the set of positive integers 
to itself is uncountable. This is a consequence of the uncountability of the real numbers between 
0 and | (see Example 5). Putting these two results together (Exercise 39) shows that there are 
uncomputable functions. 


THE CONTINUUM HYPOTHESIS We conclude this section with a brief discussion of a 
famous open question about cardinality. It can be shown that the power set of Z* and the set 
of real numbers R have the same cardinality (see Exercise 38). In other words, we know that 
|P(Z*)| = |R| = c, where c denotes the cardinality of the set of real numbers. 

An important theorem of Cantor (Exercise 40) states that the cardinality of a set is always 
less than the cardinality of its power set. Hence, |Z*| < |P(Z*)|. We can rewrite this as No < 
2%, using the notation 2!5! to denote the cardinality of the power set of the set S. Also, note that 
the relationship |P(Z*)| = |R| can be expressed as 2% = ¢. 

This leads us to the continuum hypothesis, which asserts that there is no cardinal number 
X between N, and c. In other words, the continuum hypothesis states that there is no set A such 
that No, the cardinality of the set of positive integers, is less than |A| and |A] is less than c, the 
cardinality of the set of real numbers. It can be shown that the smallest infinite cardinal numbers 
form an infinite sequence Ny < N; < N, < .... If we assume that the continuum hypothesis is 
true, it would follow that ¢ = N}, so that 2% = N}. 
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The continuum hypothesis was stated by Cantor in 1877. He labored unsuccessfully to prove 
it, becoming extremely dismayed that he could not. By 1900, settling the continuum hypothesis 
was considered to be among the most important unsolved problems in mathematics. It was the 
first problem posed by David Hilbert in his 1900 list of open problems in mathematics. 

The continuum hypothesis is still an open question and remains an area for active research. 
However, it has been shown that it can be neither proved nor disproved under the standard set 
theory axioms in modern mathematics, the Zermelo-Fraenkel axioms. The Zermelo-Fraenkel 
axioms were formulated to avoid the paradoxes of naive set theory, such as Russell’s paradox, 
but there is much controversy whether they should be replaced by some other set of axioms for 


set theory. 


Exercises 


1. Determine whether each of these sets is finite, countably 


infinite, or uncountable. For those that are countably in- 

finite, exhibit a one-to-one correspondence between the 

set of positive integers and that set. 

a) the negative integers 

b) the even integers 

c) the integers less than 100 

d) the real numbers between 0 and ; 

e) the positive integers less than 1,000,000,000 

f) the integers that are multiples of 7 

. Determine whether each of these sets is finite, countably 

infinite, or uncountable. For those that are countably in- 

finite, exhibit a one-to-one correspondence between the 

set of positive integers and that set. 

a) the integers greater than 10 

b) the odd negative integers 

c) the integers with absolute value less than 1,000,000 

d) the real numbers between 0 and 2 

e) the set A x Z* where A = {2, 3} 

f) the integers that are multiples of 10 

. Determine whether each of these sets is countable or un- 

countable. For those that are countably infinite, exhibit 

a one-to-one correspondence between the set of positive 

integers and that set. 

a) all bit strings not containing the bit 0 

b) all positive rational numbers that cannot be written 
with denominators less than 4 

c) the real numbers not containing 0 in their decimal rep- 
resentation 

d) the real numbers containing only a finite number of 
1s in their decimal representation 

. Determine whether each of these sets is countable or un- 

countable. For those that are countably infinite, exhibit 

a one-to-one correspondence between the set of positive 

integers and that set. 

a) integers not divisible by 3 

b) integers divisible by 5 but not by 7 

c) the real numbers with decimal representations con- 
sisting of all 1s 

d) the real numbers with decimal representations of all 
Is or 9s 


*9, 


10. 


11. 


. Show that a finite group of guests arriving at Hilbert’s 


fully occupied Grand Hotel can be given rooms without 
evicting any current guest. 


. Suppose that Hilbert’s Grand Hotel is fully occupied, but 


the hotel closes all the even numbered rooms for mainte- 
nance. Show that all guests can remain in the hotel. 


. Suppose that Hilbert’s Grand Hotel is fully occupied on 


the day the hotel expands to a second building which also 
contains a countably infinite number of rooms. Show that 
the current guests can be spread out to fill every room of 
the two buildings of the hotel. 


. Show that a countably infinite number of guests arriv- 


ing at Hilbert’s fully occupied Grand Hotel can be given 
rooms without evicting any current guest. 

Suppose that a countably infinite number of buses, each 
containing a countably infinite number of guests, arrive 
at Hilbert’s fully occupied Grand Hotel. Show that all the 
arriving guests can be accommodated without evicting 
any current guest. 

Give an example of two uncountable sets A and B such 
that A — B is 

a) finite. 

b) countably infinite. 

c) uncountable. 

Give an example of two uncountable sets A and B such 
that A N B is 

a) finite. 

b) countably infinite. 

c) uncountable. 


. Show that if A and B are sets and A C B then |A| < |B|. 
. Explain why the set A is countable if and only if |A| < 


|Z*]. 


. Show that if A and B are sets with the same cardinality, 


then |A| < |B] and |B| < JA]. 


. Show that if A and B are sets, A is uncountable, and 


A C B, then B is uncountable. 


. Show that a subset of a countable set is also countable. 
. If A is an uncountable set and B is a countable set, must 


A — B be uncountable? 


. Show that if A and B are sets |A| = |B|, then |P(A)| = 


|P(B)I. 


. Show that if A, B, C, and D are sets with |A| = |B| and 


|C| = |D], then |A x C| = |B x D|. 


20. 
21. 


22. 


23. 


24. 


25. 


26. 


*27. 


28. 
*29. 
*30. 


*31. 


*32. 


33. 


34. 


35. 


*36. 


*37. 


Show that if |A| = |B] and |B| = |C], then |A| = |C]. 

Show that if A, B, and C are sets such that |A| < |B| and 

|B| < |C], then |A| < |C]. 

Suppose that A is a countable set. Show that the set B is 

also countable if there is an onto function f from A to B. 

Show that if A is an infinite set, then it contains a count- 

ably infinite subset. 

Show that there is no infinite set A such that |A| < |Z*| = 

No. 

Prove that if it is possible to label each element of an infi- 

nite set S with a finite string of keyboard characters, from 

a finite list characters, where no two elements of S have 

the same label, then S is a countably infinite set. 

Use Exercise 25 to provide a proof different from that 

in the text that the set of rational numbers is countable. 

[Hint: Show that you can express a rational number as a 

string of digits with a slash and possibly a minus sign.] 

Show that the union of a countable number of countable 

sets is countable. 

Show that the set Z+ x Z* is countable. 

Show that the set of all finite bit strings is countable. 

Show that the set of real numbers that are solutions of 

quadratic equations ax? + bx +c = 0, where a, b, and c 

are integers, is countable. 

Show that Z* x Z* is countable by showing that the 

polynomial function f : Z* x Z* > Z* with f(m, n) = 

(m+n —2)(m +n — 1)/2 +m is one-to-one and onto. 

Show that when you substitute (3n + 1)* for each occur- 

rence of n and (3m + 1} for each occurrence of m in the 

right-hand side of the formula for the function f(m, n) in 

Exercise 31, you obtain a one-to-one polynomial func- 

tion Z x Z > Z. It is an open question whether there is a 

one-to-one polynomial function Q x Q > Q. 

Use the Schréder-Bernstein theorem to show that (0, 1) 

and [0, 1] have the same cardinality. 

Show that (0, 1) and R have the same cardinality by 

a) showing that f(x) = ES 
to R. 

b) using the Schröder-Bernstein theorem. 


is a bijection from (0, 1) 


Show that there is no one-to-one correspondence from the 
set of positive integers to the power set of the set of positive 
integers. [Hint: Assume that there is such a one-to-one 
correspondence. Represent a subset of the set of positive 
integers as an infinite bit string with ith bit 1 if 7 belongs 
to the subset and 0 otherwise. Suppose that you can list 
these infinite strings in a sequence indexed by the positive 
integers. Construct a new bit string with its ith bit equal to 
the complement of the ith bit of the ith string in the list. 
Show that this new bit string cannot appear in the list.] 
Show that there is a one-to-one correspondence from the 
set of subsets of the positive integers to the set real num- 
bers between 0 and 1. Use this result and Exercises 34 and 
35 to conclude that Nọ < |P(Z*)| = |R|. [Hint: Look at 
the first part of the hint for Exercise 35.] 

Show that the set of all computer programs in a partic- 
ular programming language is countable. [Hint: A com- 


* 38, 


* 39, 


* 40. 


* AL. 
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puter program written in a programming language can be 
thought of as a string of symbols from a finite alphabet. ] 


Show that the set of functions from the positive inte- 
gers to the set {0, 1, 2,3, 4, 5, 6, 7, 8,9} is uncountable. 
[Hint: First set up a one-to-one correspondence between 
the set of real numbers between 0 and 1 and a subset of 
these functions. Do this by associating to the real number 
0.d,d, ...d,,... the function f with f(n) = d,,.] 

We say that a function is computable if there is a com- 
puter program that finds the values of this function. Use 
Exercises 37 and 38 to show that there are functions that 
are not computable. 


Show that if S is a set, then there does not exist an onto 

function f from S to P(S), the power set of S. Con- 

clude that |S| < |P(S)|. This result is known as Cantor’s 
theorem. [Hint: Suppose such a function f existed. Let 

T = {s E€ S | s £ f(s)} and show that no element s can ex- 

ist for which f(s) = T.] 

In this exercise, we prove the Schröder-Bernstein theo- 

rem. Suppose that A and B are sets where |A| < |B| and 

|B| < |A|. This means that there are injections f : A > B 

and g : B > A. To prove the theorem, we must show that 

there is a bijection h : A > B, implying that |A| = |B|. 
To build h, we construct the chain of an el- 

ement a EA. This chain contains the elements 
a, fla), g(a). FED), 8E), -.. . It also may 
contain more elements that precede a, extending the 
chain backwards. So, if there is a b € B with g(b) =a, 
then b will be the term of the chain just before a. Because 
g may not be a surjection, there may not be any such b, so 
that a is the first element of the chain. If such a b exists, 
because g is an injection, it is the unique element of B 
mapped by g to a; we denote it by g7! (a). (Note that this 
defines g7! as a partial function from B to A.) We extend 
the chain backwards as long as possible in the same way, 
adding f~! (g7! (a)), g7! (f7! (871 (a))), .... To construct 
the proof, complete these five parts. 

a) Show that every element in A or in B belongs to ex- 
actly one chain. 

b) Show that there are four types of chains: chains that 
form a loop, that is, carrying them forward from ev- 
ery element in the chain will eventually return to this 
element (type 1), chains that go backwards without 
stopping (type 2), chains that go backwards and end 
in the set A (type 3), and chains that go backwards 
and end in the set B (type 4). 

c) We now define a function h : A > B. We set h(a) = 

f(a) when a belongs to a chain of type 1, 2, or 3. Show 

that we can define h(a) when a is in a chain of type 4, 

by taking h(a) = g7! (a). In parts (d) and (e), we show 

that this function is a bijection from A to B, proving 
the theorem. 

Show that / is one-to-one. (You can consider chains 

of types 1, 2, and 3 together, but chains of type 4 

separately.) 

Show that h is onto. (You need to consider chains 

of types 1, 2, and 3 together, but chains of type 4 

separately.) 


d 


ha 


e 


wm 
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Matrices 


2.6.1 Introduction 


Matrices are used throughout discrete mathematics to express relationships between elements 
in sets. In subsequent chapters we will use matrices in a wide variety of models. For instance, 
matrices will be used in models of communications networks and transportation systems. Many 
algorithms will be developed that use these matrix models. This section reviews matrix arith- 
metic that will be used in these algorithms. 


Definition 1 A matrix is a rectangular array of numbers. A matrix with m rows and n columns is called an 
m Xn matrix. The plural of matrix is matrices. A matrix with the same number of rows as 
columns is called square. Two matrices are equal if they have the same number of rows and 
the same number of columns and the corresponding entries in every position are equal. 


1 1 
EXAMPLE 1 The matrix|0 2)is a3 x2 matrix. 
1 3 < 


We now introduce some terminology about matrices. Boldface uppercase letters will be 
used to represent matrices. 


Definition2 Let mand n be positive integers and let 


@ig Ge x Cip 
Coi Gap An 
AS 
amı Am2 GHE amn 
The ith row of A is the 1 x n matrix [a; dj, ..., Ain]. The jth column of A is the m X 1 matrix 
a 


mj 


The (i, j)th element or entry of A is the element a; that is, the number in the ith row and 
jth column of A. A convenient shorthand notation for expressing the matrix A is to write 
A= [a;l], which indicates that A is the matrix with its (i, j)th element equal to aj. 


Definition 3 


EXAMPLE 2 


Definition 4 


EXAMPLE 3 
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2.6.2 Matrix Arithmetic 


The basic operations of matrix arithmetic will now be discussed, beginning with a definition of 
matrix addition. 


Lera = [a;] and B = [bil be m Xn matrices. The sum of A and B, denoted by A + B, is 
the m x n matrix that has a, + b; as its (i, /)th element. In other words, A + B = [a; + bil. 


The sum of two matrices of the same size is obtained by adding elements in the corresponding 
positions. Matrices of different sizes cannot be added, because such matrices will not both have 
entries in some of their positions. 


1 0 -1 3 4 -1 4 4 -2 
We haveļ|2 2 -—3ļ|+| 1 -3 0ļ=]|3 -1 —3}. 
3 4 (0) —1 1 2 2 5 2 < 


We now discuss matrix products. A product of two matrices is defined only when the num- 
ber of columns in the first matrix equals the number of rows of the second matrix. 


Let A be an m X k matrix and B be a k X n matrix. The product of A and B, denoted by AB, is 
the m X n matrix with its (i, j)th entry equal to the sum of the products of the corresponding 
elements from the ith row of A and the jth column of B. In other words, if AB = [cyl then 


Ci = ajbi; F anb; ap 080 SP airb: 


In Figure 1 the colored row of A and the colored column of B are used to compute the element 
c, of AB. The product of two matrices is not defined when the number of columns in the first 
matrix and the number of rows in the second matrix are not the same. 

We now give some examples of matrix products. 


Let 
TE 2 4 
A= and B=|1 1 
3 1 0 3 0 
0 2 2 
Find AB if it is defined. 
a&i an aik 
a an Ak bı bp bij bin Ci C2 Cin 
: 2 |} bo) dy. by ban |n “x Con 
4i an Gig : : : Ci 
: : i by bp Dy Din Cmi Cm =- Cmn 
amı Am2 amk 


FIGURE 1 The product of A = [a;] and B = [b;;]. 
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Extra 
Examples 


EXAMPLE 4 


Definition 5 


Solution: Because A is a4 x 3 matrix and B is a3 X 2 matrix, the product AB is defined and is a 
4 x 2 matrix. To find the elements of AB, the corresponding elements of the rows of A and the 
columns of B are first multiplied and then these products are added. For instance, the element in 
the (3, 1)th position of AB is the sum of the products of the corresponding elements of the third 
row of A and the first column of B; namely, 3-2 + 1-1+0-3 = 7. When all the elements of 
AB are computed, we see that 


14 4 
8 9 
AB = 7 43I° 
8 2 < 


Although matrix multiplication is associative, as can easily be proved using the associativity 
of addition and multiplication of real numbers, matrix multiplication is not commutative. That 
is, if A and B are two matrices, it is not necessarily true that AB and BA are the same. In fact, it 
may be that only one of these two products is defined. For instance, if A is 2 x 3 and B is 3 x 4, 
then AB is defined and is 2 x 4; however, BA is not defined, because it is impossible to multiply 
a3 x4 matrix and a 2 x 3 matrix. 

In general, suppose that A is an m X n matrix and B is an r X s matrix. Then AB is defined 
only when n = r and BA is defined only when s = m. Moreover, even when AB and BA are 
both defined, they will not be the same size unless m = n = r = s. Hence, if both AB and BA 
are defined and are the same size, then both A and B must be square and of the same size. 
Furthermore, even with A and B both n X n matrices, AB and BA are not necessarily equal, as 
Example 4 demonstrates. 


Let 


Does AB = BA? 


Solution: We find that 


3 2 4 3 
ap =|) | and Ba = |; AR 


Hence, AB + BA. < 


2.6.3 Transposes and Powers of Matrices 


We now introduce an important matrix with entries that are zeros and ones. 


The identity matrix of order n is the n X n matrix I, = [ô;], (the Kronecker delta) where 
6, = l if i = j and ô; = 0 if i # j. Hence, 


L O a A 
© il 


i 


Definition 6 


EXAMPLE 5 


Definition 7 


FIGURE2 A 
symmetric matrix. 


EXAMPLE 6 
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Multiplying a matrix by an appropriately sized identity matrix does not change this matrix. In 
other words, when A is an m x n matrix, we have 


AI, =1,A =A. 
Powers of square matrices can be defined because matrix multiplication is associative. When A 


is an n X n matrix, we have 


n 


A° =I, A’ = AAA---A. 
——S” 
r times 


The operation of interchanging the rows and columns of a square matrix arises in many 
contexts. 


cerAe— [a;l] be an m X n matrix. The transpose of A, denoted by A‘, is the n x m matrix 
obtained by interchanging the rows and columns of A. In other words, if A‘ = [b;;], then 
b; = a; for i = I, soon th EN = IL, B aeaii 


AUA 


1 
The transpose of the matrix is is the matrix | 2 
4 5 6 3 


< 


Matrices that do not change when their rows and columns are interchanged are often im- 
portant. 


A square matrix A is called symmetric if A = A‘. Thus, A = [a,] is symmetric if a; = a;; for 
alliandj with 1 <i<nand1<j<n. 


Note that a matrix is symmetric if and only if it is square and it is symmetric with respect to its 
main diagonal (which consists of entries that are in the ith row and ith column for some i). This 
symmetry is displayed in Figure 2. 


is symmetric. 


oro 


1 1 
The matrix]1 0O 
0 1 


2.6.4 Zero—One Matrices 


A matrix all of whose entries are either 0 or 1 is called a zero—one matrix. Zero—one matrices 
are often used to represent discrete structures, as we will see in Chapters 9 and 10. Algorithms 
using these structures are based on Boolean arithmetic with zero—one matrices. This arithmetic 
is based on the Boolean operations A and v, which operate on pairs of bits, defined by 


kaha 1 ifb =b,=1 
l 27 10 otherwise, 


kyke 1 ifb =lorb,=1 
f >" \ 0 otherwise. 
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Definition 8 — Let A = [a;] and B = [b,] be m x n zero—one matrices. Then the join of A and B is the zero- 
one matrix with (i, j)th entry a; V b;. The join of A and B is denoted by A v B. The meet of 
A and B is the zero—one matrix with (i, j)th entry a; A b;. The meet of A and B is denoted 
by AAB. 


EXAMPLE 7 Find the join and meet of the zero—one matrices 
1 0 1 0 1 0 
alo 1 T B= (i 1 Ji 
Solution: We find that the join of A and B is 


—|t a4 
Sj a Of 


“bt of ‘ 


lvO Ovl Il1v0 


avB= hovi Ivi 0v0 


The meet of A and B is 


1A0 OAL 1^0 


anB= lohi 1A1 0A0 


We now define the Boolean product of two matrices. 


Definition 9 Let A = [a,] be an m x k zero—one matrix and B = [,] be a k x n zero-one matrix. Then 
the Boolean product of A and B, denoted by A © B, is the m x n matrix with (i, /)th entry c; 
where 


eS (aj A b;;) V (an A bo;) VEEN. (ix A by). 


Note that the Boolean product of A and B is obtained in an analogous way to the ordinary prod- 
uct of these matrices, but with addition replaced with the operation V and with multiplication 
replaced with the operation ^. We give an example of the Boolean products of matrices. 


EXAMPLE 8 Find the Boolean product of A and B, where 


a 


1 0 
A=]0 1j, B=| . 
1 0 O 1 1 


Solution: The Boolean product A © B is given by 


GALDVOAN) GADVOAL CGADVOAIL 
AOB=/OA1V(ULA0N) OALDVUAL OAdVvVUAL 
GALDVOAIN) GADVOAIL CGADVOAI) 


1v0 1v0 Ovo 
Ovo Ovl Ovi 
1v0 1v0 Ovo 


1 1 0 
=|0 1 1j 
1 1 0 < 
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We can also define the Boolean powers of a square zero—one matrix. These powers will 
be used in our subsequent studies of paths in graphs, which are used to model such things as 
communications paths in computer networks. 


Definition 10 


Let A be a square zero—one matrix and let r be a positive integer. The rth Boolean power of 


A is the Boolean product of r factors of A. The rth Boolean product of A is denoted by A". 


Hence, 


A"! = AOAOAO.--- OA. 
ee 


r times 


(This is well defined because the Boolean product of matrices is associative.) We also define 


AÙ! to be I. 


0 0 1 
EXAMPLE 9 LetA=|1 0 OJ}. Find A™ for all positive integers n. 
1 1 0 


Solution: We find that 


We also find that 
1 0 1 1 1 1 
API=APIOA=|1 1 Of, AM=APIOAH=]1 0 1j. 
1 1 1 1 1 1 
Additional computation shows that 
1 
ASl=}1 
1 1 1 
The reader can now see that A!”! = Al! for all positive integers n with n > 5. < 
Exercises 
1 1 1 3 2. Find A + B, where 
1. Le A=|2 0 4 6]. 1 Oo 4 -1 3 5 
1 1 3 7 a) A=|-—1 2 2|,B=| 2 2 -3). 
a) What size is A? 0 -2 -3 2 -3 0 
b) What is the third column of A? _{-l 05 6 _|-3 9 -3 4 
MASIA at a a= oe a, Bl 
c) What is the second row of A? ine ~ = 


d) What is the element of A in the (3, 2)th position? 
e) What is A’? 


3. Find AB if 


21 0 4 
Dach Jael R 
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10. 


11. 


12. 


13. 


14. 


1 -l 
b) A=|0 i]. B=]; k a 
2 3 
4 -3 
3 -l1 —1 3 2 -2 
mE: PEE -1 4 ze 
—1 5 
Find the product AB, where 
1 (0) 1 (0) 1 -l 
a) A=| 0 1 1|,B=] 1 1 0 
—1 1 (0) —1 1 
1 -3 (0) 1 -1 2 3 
b) A=] 1 2 2),B=|-1 0 3 -Il}. 
2 1 -l -3 -2 0 2 
0 -l 
4 -1 2 3 0 
c) A=| 7 2 B=| | 
4 3 —2 03 4 1 


. Find a matrix A such that 


2 3 3 0 
AeA 
[Hint: Finding A requires that you solve systems of linear 
equations.] 


. Find a matrix A such that 


. Let A be an m x n matrix and let 0 be the m X n matrix 


that has all entries equal to zero. Show that A = 0 + A = 
A +0. 


. Show that matrix addition is commutative; that is, 


show that if A and B are both mxn matrices, then 
A+B=B+A. 


. Show that matrix addition is associative; that is, show 


that if A, B, and C are all mxn matrices, then 

A+(B+C)=(A+B)+C. 

Let A be a 3 x 4 matrix, B be a4 x 5 matrix, and C bea 

4 x 4 matrix. Determine which of the following products 

are defined and find the size of those that are defined. 

a) AB b) BA c) AC 

d) CA e) BC f) CB 

What do we know about the sizes of the matrices A and 

B if both of the products AB and BA are defined? 

In this exercise we show that matrix multiplication is dis- 

tributive over matrix addition. 

a) Suppose that A and B are m x k matrices and that C 
is a k Xn matrix. Show that (A + B)C = AC + BC. 

b) Suppose that C is an m x k matrix and that A and B 
are k X n matrices. Show that C(A + B) = CA + CB. 

In this exercise we show that matrix multiplication is 

associative. Suppose that A is an m Xp matrix, B is 

a pxXk matrix, and C is a kxn matrix. Show that 

A(BC) = (AB)C. 

The n Xn matrix A = [aj] is called a diagonal matrix 

if a, = 0 when i # j. Show that the product of two n x n 


15. 


16. 
17. 


diagonal matrices is again a diagonal matrix. Give a sim- 
ple rule for determining this product. 


Let 


Find a formula for A”, whenever n is a positive integer. 
Show that (A‘)' = A. 

Let A and B be two n X n matrices. Show that 

a) (A + BY = A' + B’. 

b) (ABY = B'A’. 


If A and B are n Xn matrices with AB = BA = I,, then B 
is called the inverse of A (this terminology is appropriate be- 
cause such a matrix B is unique) and A is said to be invertible. 
The notation B = A! denotes that B is the inverse of A. 


18. 


19. 


20. 


21. 


22. 


Show that 
2 3 -1 
1 2 1 
-1 -l 3 
is the inverse of 
7 -8 5 
—4 5 -3]. 
1 -l 1 


Let A be the 2 x 2 matrix 
a b 
c d|’ 


Show that if ad — bc # 0, then 


A= 


d —b 
Avs ad—be ad- bc 
=c a 
ad—bc ad- bc 
Let 
-1 2 
aT 
a) Find A~!. [Hint: Use Exercise 19.] 
b) Find A?. 


c) Find (A7!)3. 

d) Use your answers to (b) and (c) to show that (A7!)? 
is the inverse of A?. 

Let A be an invertible matrix. Show that (A”)~! = (A7!)" 

whenever n is a positive integer. 

Let A be a matrix. Show that the matrix AA’ is symmet- 


ric. [Hint: Show that this matrix equals its transpose with 
the help of Exercise 17b.] 


23. Suppose that A is an n X n matrix where n is a positive 
integer. Show that A + A‘ is symmetric. 
24. a) Show that the system of simultaneous linear equa- 
tions 


411X] + AyQXy +++ +.44,x, = d, 


Ay 1X1 + aX + = + Ay, X, = Dy 


aX + AngXqy +s + Ay X, = bn: 
in the variables x,,x5,...,x, can be expressed as 
AX = B, where A = [a;l], X isann X | matrix with x; 
the entry in its ith row, and B is an n X 1 matrix with 
b; the entry in its ith row. 

b) Show that if the matrix A = [aj] is invertible (as de- 
fined in the preamble to Exercise 18), then the solu- 
tion of the system in part (a) can be found using the 
equation X = A7!B. 

25. Use Exercises 18 and 24 to solve the system 


7x, — 8x, + 5x3; =5 
—4x + 5x — 3x3 = —3 


x= +43 =0 


26. Let 
i A _fo 1 
A= k l and B= f o 
Find 
a) AvB. b) AAB. c) AOB. 
27. Let 
1 0O 1 0O 1 1 
A=]1 1 0 and B=|1 0 1}. 
0 0 1 101 
Find 
a) AVB. b) AAB. c) AOB. 
Key Terms and Results 
TERMS 


set: an unordered collection of distinct objects 
axiom: a basic assumption of a theory 
paradox: a logical inconsistency 

element, member of a set: an object in a set 


roster method: a method that describes a set by listing its 
elements 

set builder notation: the notation that describes a set by stat- 
ing a property an element must have to be a member 


multiset: an unordered collection of objects where objects can 
occur multiple times 
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28. Find the Boolean product of A and B, where 


1 0 1 ; 7 
A=|0 1 0 1 and B= ; 
1 1 1 1 E 
1 0 
29. Let 
1 0 
A=|1 0 1 
0 1 
Find 
a) A, b) ABI, 


ce) AVA y ABI 

30. Let A be a zero—one matrix. Show that 
a) AVA=A. b) AAA=A. 

31. In this exercise we show that the meet and join opera- 
tions are commutative. Let A and B be m xn zero—one 
matrices. Show that 
a) AVB=BVA. b) BAA=AAB. 

32. In this exercise we show that the meet and join opera- 
tions are associative. Let A, B, and C be m Xx n zero—one 
matrices. Show that 
a) (AVB)VC=Av(BVC). 

b) AAB)AC=AA (BAC). 

33. We will establish distributive laws of the meet over the 
join operation in this exercise. Let A, B, and C be m x n 
zero—one matrices. Show that 
a) AV(BAC)=(AVB)A(A VOC). 

b) AA(BVC)=(AAB)V(AAC). 

34. Let A be an n Xn zero—one matrix. Let I be the nxn 
identity matrix. Show that A OI =IOA =A. 

35. In this exercise we will show that the Boolean product 
of zero—one matrices is associative. Assume that A is an 
m x p zero—one matrix, B is a p X k zero—one matrix, and 
C is a k Xn zero—one matrix. Show that A © (B © C) = 
(AOB)OC. 


Ø (empty set, null set): the set with no members 

universal set: the set containing all objects under considera- 
tion 

Venn diagram: a graphical representation of a set or sets 

S = T (set equality): S and T have the same elements 


S C T (S is a subset of T): every element of S is also an ele- 
ment of T 


S CT (S is a proper subset of T): S is a subset of T and S 4 T 

finite set: a set with n elements, where n is a nonnegative 
integer 

infinite set: a set that is not finite 

|S| (the cardinality of S): the number of elements in S 
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P(S) (the power set of S): the set of all subsets of $ 


A U B (the union of A and B): the set containing those ele- 
ments that are in at least one of A and B 


A N B (the intersection of A and B): the set containing those 
elements that are in both A and B. 


A — B (the difference of A and B): the set containing those 
elements that are in A but not in B 


A (the complement of A): the set of elements in the universal 
set that are not in A 


A @ B (the symmetric difference of A and B): the set con- 
taining those elements in exactly one of A and B 


membership table: a table displaying the membership of el- 
ements in sets 


function from A to B : an assignment of exactly one element 
of B to each element of A 


domain of f : the set A, where f is a function from A to B 
codomain of f : the set B, where f is a function from A to B 
b is the image of a under f: b = f(a) 

ais a preimage of b under f: f(a) = b 

range of f: the set of images of f 


onto function, surjection: a function from A to B such that 
every element of B is the image of some element in A 


one-to-one function, injection: a function such that the im- 
ages of elements in its domain are distinct 


one-to-one correspondence, bijection: a function that is both 
one-to-one and onto 


inverse of f: the function that reverses the correspondence 
given by f (when f is a bijection) 
f °g (composition of f and g): the function that assigns f(g(x)) 
tox 
|x] (floor function): the largest integer not exceeding x 


[x] (ceiling function): the smallest integer greater than or 
equal to x 

partial function: an assignment to each element in a subset of 
the domain a unique element in the codomain 

sequence: a function with domain that is a subset of the set of 
integers 

geometric progression: a sequence of the form a, ar, ar’, ..., 
where a and r are real numbers 

arithmetic progression: a sequence of the form a, a+d, 
a+ 2d, ..., where a and d are real numbers 

string: a finite sequence 


empty string: a string of length zero 


Review Questions 


1. Explain what it means for one set to be a subset of another 
set. How do you prove that one set is a subset of another 
set? 

2. What is the empty set? Show that the empty set is a subset 
of every set. 


3. a) Define |S|, the cardinality of the set S. 


recurrence relation: a equation that expresses the nth term a,, 
of a sequence in terms of one or more of the previous terms 
of the sequence for all integers n greater than a particular 
integer 

Dd). a the sum a, +a +- +a, 

IL- a;: the product a,a, + a, 

cardinality: two sets A and B have the same cardinality if there 
is a one-to-one correspondence from A to B 

countable set: a set that either is finite or can be placed in 
one-to-one correspondence with the set of positive integers 

uncountable set: a set that is not countable 

No (aleph null): the cardinality of a countable set 

c: the cardinality of the set of real numbers 


Cantor diagonalization argument: a proof technique used to 
show that the set of real numbers is uncountable 


computable function: a function for which there is a com- 
puter program in some programming language that finds 
its values 


uncomputable function: a function for which no computer 
program in a programming language exists that finds its 
values 

continuum hypothesis: the statement that no set A exists such 
that Ny < JA] < ¢ 

matrix: a rectangular array of numbers 

matrix addition: see page 188 

matrix multiplication: see page 189 

I, (identity matrix of order n): the n X n matrix that has en- 
tries equal to 1 on its diagonal and Os elsewhere 

A! (transpose of A): the matrix obtained from A by inter- 
changing the rows and columns 


symmetric matrix: a matrix is symmetric if it equals its trans- 
pose 


zero—one matrix: a matrix with each entry equal to either 0 
or 1 


A V B (the join of A and B): see page 191 
A AB (the meet of A and B): see page 191 
A © B (the Boolean product of A and B): see page 192 


RESULTS 


The set identities given in Table 1 in Section 2.2 
The summation formulae in Table 2 in Section 2.4 
The set of rational numbers is countable. 

The set of real numbers is uncountable. 


b) Give a formula for |A U B|, where A and B are sets. 
4. a) Define the power set of a set S. 
b) When is the empty set in the power set of a set $? 


c) How many elements does the power set of a set S with 
n elements have? 


a) Define the union, intersection, difference, and sym- 
metric difference of two sets. 

b) What are the union, intersection, difference, and sym- 
metric difference of the set of positive integers and the 
set of odd integers? 


a) Explain what it means for two sets to be equal. 

b) Describe as many of the ways as you can to show that 
two sets are equal. 

c) Show in at least two different ways that the sets A — 
(B N C) and (A — B) U (A — C) are equal. 

Explain the relationship between logical equivalences 

and set identities. 


a) Define the domain, codomain, and range of a function. 

b) Let f(n) be the function from the set of integers to the 
set of integers such that f(n) = n? + 1. What are the 
domain, codomain, and range of this function? 


a) Define what it means for a function from the set of 
positive integers to the set of positive integers to be 
one-to-one. 

b) Define what it means for a function from the set of 
positive integers to the set of positive integers to be 
onto. 

c) Give an example of a function from the set of posi- 
tive integers to the set of positive integers that is both 
one-to-one and onto. 

d) Give an example of a function from the set of positive 
integers to the set of positive integers that is one-to- 
one but not onto. 


Supplementary Exercises 


1. 


Let A be the set of English words that contain the letter 

x, and let B be the set of English words that contain the 

letter g. Express each of these sets as a combination of A 

and B. 

a) The set of English words that do not contain the 
letter x. 

b) The set of English words that contain both an x 
and a q. 

c) The set of English words that contain an x but not a q. 

d) The set of English words that do not contain either an 
xoraq. 

e) The set of English words that contain an x or a q, but 
not both. 


. Show that if A is a subset of B, then the power set of A is 


a subset of the power set of B. 


. Suppose that A and B are sets such that the power set of 


A is a subset of the power set of B. Does it follow that A 
is a subset of B? 


. Let E denote the set of even integers and O denote the 


set of odd integers. As usual, let Z denote the set of all 
integers. Determine each of these sets. 


a)EUO b)EnNO c)Z—-E d)Z-—O 


. Show that if A and B are sets, then A — (A — B) =A N B. 


10. 


11. 


12. 


13. 


14. 


15. 
16. 
17. 


18. 


12. 


13. 
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e) Give an example of a function from the set of posi- 
tive integers to the set of positive integers that is not 
one-to-one but is onto. 

f) Give an example of a function from the set of positive 
integers to the set of positive integers that is neither 
one-to-one nor onto. 

a) Define the inverse of a function. 

b) When does a function have an inverse? 

c) Does the function f(n) = 10 — n from the set of inte- 
gers to the set of integers have an inverse? If so, what 
is it? 

a) Define the floor and ceiling functions from the set of 
real numbers to the set of integers. 

b) For which real numbers x is it true that |x] = [x]? 

Conjecture a formula for the terms of the sequence that 

begins 8, 14, 32, 86, 248 and find the next three terms of 

your sequence. 

Suppose that a, =a,_, — 5 for n = 1,2,.... Find a for- 

mula for a,,. 

What is the sum of the terms of the geometric progression 

a+ar + +ar” whenr # 1? 


Show that the set of odd integers is countable. 
Give an example of an uncountable set. 
Define the product of two matrices A and B. When is this 
product defined? 
Show that matrix multiplication is not commutative. 

. Let A and B be sets. Show that A C B if and only if 
ANB=A. 

. Let A, B, and C be sets. Show that (A — B) — C is not nec- 
essarily equal to A — (B — C). 

. Suppose that A, B, and C are sets. Prove or disprove that 
(A-—B)-C=(A-O)-B. 

. Suppose that A, B, C, and D are sets. Prove or disprove 
that (A — B) — (C — D) = (A-C)-(B-D). 

. Show that if A and B are finite sets, then |A N B| < 
|A U B|. Determine when this relationship is an equality. 

. Let A and B be sets in a finite universal set U. List the 
following in order of increasing size. 
a) |A|, IA U BI, |AN BI, [U], IØ] 
b) |A - B|, |A @ BI, |A| + |B], IA U BI, |9| 
Let A and B be subsets of the finite universal set U. Show 
that |AN B| = |U] - |A| — |B| + |A nB]. 
Let f and g be functions from {1, 2, 3, 4} to {a, b, c,d} 
and from {a,b,c,d} to {1,2,3,4}, respectively, with 
fC) = d, f2) = c, fG) = a, and f(4) = b, and g(a) = 2, 


g(b) = 1, g(c) = 3, and g(d) = 2. 

a) Is f one-to-one? Is g one-to-one? 

b) Is f onto? Is g onto? 

c) Does either f or g have an inverse? If so, find this in- 
verse. 
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14. 


15. 


Suppose that f is a function from A to B where A and B 
are finite sets. Explain why |f(S)| < |S] for all subsets S 
of A. 

Suppose that f is a function from A to B where A and B 
are finite sets. Explain why [f(S)| = |S| for all subsets S 
of A if and only if f is one-to-one. 


Suppose that f is a function from A to B. We define the func- 
tion Sp from P(A) to P(B) by the rule SX) = f(X) for each 
subset X of A. Similarly, we define the function Spi from P(B) 
to P(A) by the rule Spi(Y) = f- (Y) for each subset Y of B. 
Here, we are using Definition 4, and the definition of the in- 
verse image of a set found in the preamble to Exercise 44, both 
in Section 2.3. 


* 16. 


17. 


18. 


19. 


20. 


21. 


22. 
23. 


24. 
25. 


26. 


* 27. 


Suppose that f is a function from the set A to the set B. 

Prove that 

a) if f is one-to-one, then S; is a one-to-one function 
from P(A) to P(B). 

b) if f is onto function, then S is an onto function from 
P(A) to P(B). 

c) iff is onto function, then Spa is a one-to-one function 
from P(B) to P(A). 

d) if f is one-to-one, then S;-1 is an onto function from 
P(B) to P(A). 

e) if f is a one-to-one correspondence, then Sy is a one- 
to-one correspondence from P(A) to P(B) and Spot 
is a One-to-one correspondence from P(B) to P(A). 
[Hint: Use parts (a)—(d).] 

Prove that if f and g are functions from A to B and $; = S, 

(using the definition in the preamble to Exercise 16), then 

F(x) = g(x) forall x E€ A. 

Show that if n is an integer, then n = [n/2] + |[n/2]. 

For which real numbers x and y is it true that |x + y] = 

lx] + Ly]? 

For which real numbers x and y is it true that [x + y] = 

[x] + [y]? 

For which real numbers x and y is it true that [x + y] = 

[x] + Ly]? 

Prove that |n/2] [n/2] = |n?/4] for all integers n. 

Prove that if mis an integer, then |x] + |m- x] =m- 1, 

unless x is an integer, in which case, it equals m. 

Prove that if xis areal number, then | |x/2|/2] = |x/4]. 


Prove that if n is an odd integer, then [n?/4] = 
(n? + 3)/4. 

Prove that if m and n are positive integers and x is a real 
number, then 


|==] S Eat 


Prove that if m is a positive integer and x is a real number, 
then 


[mx] = |x] +|x+—| + s+ =| ERA 
F [x+ mt 


* 28. 


29. 


* 30. 


*31, 


32. 


33. 


34. 


+= 35. 


** 36, 


37. 


38. 


39. 


40. 


41. 


We define the Ulam numbers by setting vu, = 1 and u, = 
2. Furthermore, after determining whether the integers 
less than n are Ulam numbers, we set n equal to the next 
Ulam number if it can be written uniquely as the sum of 
two different Ulam numbers. Note that u; = 3, u4 = 4, 
us = 6, and uç = 8. 

a) Find the first 20 Ulam numbers. 


b) Prove that there are infinitely many Ulam numbers. 


Determine the value of ne a (The notation used here 


for products is defined in the preamble to Exercise 45 in 
Section 2.4.) 


Determine a rule for generating the terms of the sequence 
that begins 1, 3, 4, 8, 15, 27, 50, 92, ..., and find the next 
four terms of the sequence. 


Determine a rule for generating the terms of the se- 
quence that begins 2, 3, 3, 5, 10, 13, 39, 43, 172, 177, 885, 
891, ..., and find the next four terms of the sequence. 


Show that the set of irrational numbers is an uncountable 
set. 


Show that the set S is a countable set if there is a func- 
tion f from S to the positive integers such that f~!(j) is 
countable whenever j is a positive integer. 


Show that the set of all finite subsets of the set of positive 
integers is a countable set. 


Show that |R x R| = |R|. [Hint: Use the Schröder- 
Bernstein theorem to show that |(0, 1) x (0, 1)| = |(0, 1). 
To construct an injection from (0, 1) x (0, 1) to (0, 1), sup- 
pose that (x, y) € (0, 1) x (0, 1). Map (x, y) to the number 
with decimal expansion formed by alternating between 
the digits in the decimal expansions of x and y, which do 
not end with an infinite string of 9s.] 


Show that C, the set of complex numbers, has the same 
cardinality as R, the set of real numbers. 


Find A” if A is 


I ol 


Show that if A = cI, where c is a real number and I is the 
n Xn identity matrix, then AB = BA whenever B is an 
n Xx n matrix. 


Show that if A is a2 x 2 matrix such that AB = BA when- 
ever B is a 2 x 2 matrix, then A = cI, where c is a real 
number and I is the 2 x 2 identity matrix. 


Show that if A and B are invertible matrices and AB 
exists, then (AB)! = BIAT !. 


Let A be an n Xn matrix and let 0 be the n x n matrix 
all of whose entries are zero. Show that the following are 
true. 


a) AON=00A=0 
b) AVO=0VA=A 
c) AAD=0AA=0 


Computer Projects 
Write programs with the specified input and output. 


1. Given subsets A and B of a set with n elements, use bit 
strings to find A, A UB, ANB, A — B, and A @B. 


2. Given multisets A and B from the same universal set, find 
AUB,ANB,A-—B,andA + B (see Section 2.2.5). 


3. Given fuzzy sets A and B, find A, AUB, and AN B (see 
preamble to Exercise 73 of Section 2.2). 


4. Given a function f from {1, 2, ..., n} to the set of integers, 
determine whether f is one-to-one. 


5. Given a function f from {1, 2, ..., n} to itself, determine 
whether f is onto. 


Computations and Explorations 


11. 


12. 
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. Given a bijection f from the set {1, 2, ..., n} to itself, find 


f~. 


. Given an m X k matrix A and a k x n matrix B, find AB. 

. Given a square matrix A and a positive integer n, find A”. 
. Given a square matrix, determine whether it is symmetric. 
. Given two m X n Boolean matrices, find their meet and 


join. 
Given an m X k Boolean matrix A and a k x n Boolean ma- 
trix B, find the Boolean product of A and B. 


Given a square Boolean matrix A and a positive integer n, 
find AI, 


Use a computational program or programs you have written to do these exercises. 


1. Given two finite sets, list all elements in the Cartesian 
product of these two sets. 

2. Given a finite set, list all elements of its power set. 

3. Calculate the number of one-to-one functions from a set S$ 
to a set T, where S and T are finite sets of various sizes. Can 
you determine a formula for the number of such functions? 
(We will find such a formula in Chapter 6.) 

4. Calculate the number of onto functions from a set S to a 
set T, where S and T are finite sets of various sizes. Can 
you determine a formula for the number of such functions? 
(We will find such a formula in Chapter 8.) 


Writing Projects 
Respond to these with essays using outside sources. 


1. Discuss how an axiomatic set theory can be developed to 
avoid Russell’s paradox. (See Exercise 50 of Section 2.1.) 


2. Research where the concept of a function first arose, and 
describe how this concept was first used. 


3. Explain the different ways in which the Encyclopedia of 
Integer Sequences has been found useful. Also, describe 
a few of the more unusual sequences in this encyclopedia 
and how they arise. 


5. 


*6, 


Given an positive integer n, generate the first n Fibonacci 
numbers. Looking at the terms, formulate conjectures 
about them. (For instance look at their size, divisibility 
by different factors, or possible identities relating different 
terms.) 


Develop a collection of different rules for generating the 
terms of a sequence and a program for randomly selecting 
one of these rules and the particular sequence generated 
using these rules. Make this part of an interactive program 
that prompts for the next term of the sequence and deter- 
mines whether the response is the intended next term. 


. Define the recently invented EKG sequence and describe 


some of its properties and open questions about it. 


. Look up the definition of a transcendental number. Explain 


how to show that such numbers exist and how such num- 
bers can be constructed. Which famous numbers can be 
shown to be transcendental and for which famous numbers 
is it still unknown whether they are transcendental? 


. Expand the discussion of the continuum hypothesis in the 


text. 


3.1 Algorithms 


3.2 The Growth of 
Functions 


3.3 Complexity of 
Algorithms 


Algorithms 


M any problems can be solved by considering them as special cases of general problems. 
For instance, consider the problem of locating the largest integer in the sequence 101, 
12, 144, 212, 98. This is a specific case of the problem of locating the largest integer in a se- 
quence of integers. To solve this general problem we must give an algorithm, which specifies a 
sequence of steps used to solve this general problem. We will study algorithms for solving many 
different types of problems in this book. For example, in this chapter we will introduce algo- 
rithms for two of the most important problems in computer science, searching for an element in 
a list and sorting a list so its elements are in some prescribed order, such as increasing, decreas- 
ing, or alphabetic. Later in the book we will develop algorithms that find the greatest common 
divisor of two integers, that generate all the orderings of a finite set, that find the shortest path 
between nodes in a network, and for solving many other problems. 

We will also introduce the notion of an algorithmic paradigm, which provides a general 
method for designing algorithms. In particular we will discuss brute-force algorithms, which 
find solutions using a straightforward approach without introducing any cleverness. We will also 
discuss greedy algorithms, a class of algorithms used to solve optimization problems. Proofs are 
important in the study of algorithms. In this chapter we illustrate this by proving that a particular 
greedy algorithm always finds an optimal solution. 

One important consideration concerning an algorithm is its computational complexity, 
which measures the processing time and computer memory required by the algorithm to solve 
problems of a particular size. To measure the complexity of algorithms we use big-O and big- 
Theta notation, which we develop in this chapter. We will illustrate the analysis of the complex- 
ity of algorithms in this chapter, focusing on the time an algorithm takes to solve a problem. 
Furthermore, we will discuss what the time complexity of an algorithm means in practical and 
theoretical terms. 


Algorithms 


Definition 1 


3.1.1 Introduction 


There are many general classes of problems that arise in discrete mathematics. For instance: 
given a sequence of integers, find the largest one; given a set, list all its subsets; given a set 
of integers, put them in increasing order; given a network, find the shortest path between two 
vertices. When presented with such a problem, the first thing to do is to construct a model 
that translates the problem into a mathematical context. Discrete structures used in such mod- 
els include sets, sequences, and functions—structures discussed in Chapter 2—as well as such 
other structures as permutations, relations, graphs, trees, networks, and finite state machines— 
concepts that will be discussed in later chapters. 

Setting up the appropriate mathematical model is only part of the solution. To complete the 
solution, a method is needed that will solve the general problem using the model. Ideally, what 
is required is a procedure that follows a sequence of steps that leads to the desired answer. Such 
a sequence of steps is called an algorithm. 


An algorithm is a finite sequence of precise instructions for performing a computation or for 
solving a problem. 
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The term algorithm is a corruption of the name al-Khowarizmi, a mathematician of the ninth 
century, whose book on Hindu numerals is the basis of modern decimal notation. Originally, 
the word algorism was used for the rules for performing arithmetic using decimal notation. 
Algorism evolved into the word algorithm by the eighteenth century. With the growing interest 
in computing machines, the concept of an algorithm was given a more general meaning, to 
include all definite procedures for solving problems, not just the procedures for performing 
arithmetic. (We will discuss algorithms for performing arithmetic with integers in Chapter 4.) 

In this book, we will discuss algorithms that solve a wide variety of problems. In this sec- 
tion we will use the problem of finding the largest integer in a finite sequence of integers to 
illustrate the concept of an algorithm and the properties algorithms have. Also, we will describe 
algorithms for locating a particular element in a finite set. In subsequent sections, procedures 
for finding the greatest common divisor of two integers, for finding the shortest path between 
two points in a network, for multiplying matrices, and so on, will be discussed. 


EXAMPLE 1 Describe an algorithm for finding the maximum (largest) value in a finite sequence of integers. 


Extra Even though the problem of finding the maximum element in a sequence is relatively trivial, 
Examples » it provides a good illustration of the concept of an algorithm. Also, there are many instances 
where the largest integer in a finite sequence of integers is required. For instance, a university 
may need to find the highest score on a competitive exam taken by thousands of students. Or 
a sports organization may want to identify the member with the highest rating each month. 
We want to develop an algorithm that can be used whenever the problem of finding the largest 
element in a finite sequence of integers arises. 
We can specify a procedure for solving this problem in several ways. One method is simply to 
use the English language to describe the sequence of steps used. We now provide such a solution. 


Solution of Example 1: We perform the following steps. 


1. Set the temporary maximum equal to the first integer in the sequence. (The temporary 
maximum will be the largest integer examined at any stage of the procedure.) 


2. Compare the next integer in the sequence to the temporary maximum, and if it is larger 
than the temporary maximum, set the temporary maximum equal to this integer. 


3. Repeat the previous step if there are more integers in the sequence. 


4. Stop when there are no integers left in the sequence. The temporary maximum at this 
point is the largest integer in the sequence. < 


An algorithm can also be described using a computer language. However, when that 
is done, only those instructions permitted in the language can be used. This often leads to 
a description of the algorithm that is complicated and difficult to understand. Furthermore, 
because many programming languages are in common use, it would be undesirable to choose 
one particular language. So, instead of using a particular computer language to specify 
algorithms, a form of pseudocode, described in Appendix 3, will be used in this book. (We will 
also describe algorithms using the English language.) Pseudocode provides an intermediate 


ABU JA‘FAR MOHAMMED IBN MUSA AL-KHOWARIZMI (C. 780-C. 850) al-Khowarizmi, an as- 
tronomer and mathematician, was a member of the House of Wisdom, an academy of scientists in Baghdad. 
The name al-Khowarizmi means “from the town of Kowarizm,” which was then part of Persia, but is now called 
Khiva and is part of Uzbekistan. al-Khowarizmi wrote books on mathematics, astronomy, and geography. West- 
ern Europeans first learned about algebra from his works. The word algebra comes from al-jabr, part of the title 
| of his book Kitab al-jabr w’al muquabala. This book was translated into Latin and was a widely used textbook. 
His book on the use of Hindu numerals describes procedures for arithmetic operations using these numerals. 
European authors used a Latin corruption of his name, which later evolved to the word algorithm, to describe 
~ the subject of arithmetic with Hindu numerals. 
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step between an English language description of an algorithm and an implementation of 
this algorithm in a programming language. The steps of the algorithm are specified using 
instructions resembling those used in programming languages. However, in pseudocode, the 
instructions used can include any well-defined operations or statements. A computer program 
can be produced in any computer language using the pseudocode description as a starting point. 

The pseudocode used in this book is designed to be easily understood. It can serve as an 
intermediate step in the construction of programs implementing algorithms in one of a variety of 
different programming languages. Although this pseudocode does not follow the syntax of Java, 
C, C++, or any other programming language, students familiar with a modern programming 
language will find it easy to follow. A key difference between this pseudocode and code in a 
programming language is that we can use any well-defined instruction even if it would take 
many lines of code to implement this instruction. The details of the pseudocode used in the text 
are given in Appendix 3. The reader should refer to this appendix whenever the need arises. 

A pseudocode description of the algorithm for finding the maximum element in a finite 
sequence follows. 


ALGORITHM 1 Finding the Maximum Element in a Finite Sequence. 


procedure max(qy, az, ...,d,: integers) 
max := a, 
fori :=2ton 

if max < a; then max := a; 
return max{max is the largest element} 


This algorithm first assigns the initial term of the sequence, a,, to the variable max. The “for” 
loop is used to successively examine terms of the sequence. If a term is greater than the current 
value of max, it is assigned to be the new value of max. The algorithm terminates after all terms 
have been examined. The value of max on termination is the maximum element in the sequence. 

To gain insight into how an algorithm works it is useful to construct a trace that shows its 
steps when given specific input. For instance, a trace of Algorithm 1 with input 8, 4, 11, 3, 
10 begins with the algorithm setting max to 8, the value of the initial term. It then compares 4, 
the second term, with 8, the current value of max. Because 4 < 8, max is unchanged. Next, the 
algorithm compares the third term, 11, with 8, the current value of max. Because 8 < 11, max 
is set equal to 11. The algorithm then compares 3, the fourth term, and 11, the current value of 
max. Because 3 < 11, max is unchanged. Finally, the algorithm compares 10, the first term, and 
11, the current value of max. As 10 < 11, max remains unchanged. Because there are five terms, 
we have n = 5. So after examining 10, the last term, the algorithm terminates, with max = 11. 
When it terminates, the algorithms reports that 11 is the largest term in the sequence. 


PROPERTIES OF ALGORITHMS There are several properties that algorithms generally 
share. They are useful to keep in mind when algorithms are described. These properties are: 


> Input. An algorithm has input values from a specified set. 


> Output. From each set of input values an algorithm produces output values from a spec- 
ified set. The output values are the solution to the problem. 


> Definiteness. The steps of an algorithm must be defined precisely. 

> Correctness. An algorithm should produce the correct output values for each set of input 
values. 

> Finiteness. An algorithm should produce the desired output after a finite (but perhaps 
large) number of steps for any input in the set. 

> Effectiveness. It must be possible to perform each step of an algorithm exactly and in a 
finite amount of time. 
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> Generality. The procedure should be applicable for all problems of the desired form, not 
just for a particular set of input values. 


Show that Algorithm 1 for finding the maximum element in a finite sequence of integers has all 
the properties listed. 


Solution: The input to Algorithm 1 is a sequence of integers. The output is the largest integer in 
the sequence. Each step of the algorithm is precisely defined, because only assignments, a finite 
loop, and conditional statements occur. To show that the algorithm is correct, we must show that 
when the algorithm terminates, the value of the variable max equals the maximum of the terms 
of the sequence. To see this, note that the initial value of max is the first term of the sequence; as 
successive terms of the sequence are examined, max is updated to the value of a term if the term 
exceeds the maximum of the terms previously examined. This (informal) argument shows that 
when all the terms have been examined, max equals the value of the largest term. (A rigorous 
proof of this requires the use of mathematical induction, a proof technique developed in Section 
5.1.) The algorithm uses a finite number of steps, because it terminates after all the integers in 
the sequence have been examined. The algorithm can be carried out in a finite amount of time 
because each step is either a comparison or an assignment, there are a finite number of these 
steps, and each of these two operations takes a finite amount of time. Finally, Algorithm 1 is 
general, because it can be used to find the maximum of any finite sequence of integers. < 


3.1.2 Searching Algorithms 


The problem of locating an element in an ordered list occurs in many contexts. For instance, 
a program that checks the spelling of words searches for them in a dictionary, which is just an 
ordered list of words. Problems of this kind are called searching problems. We will discuss 
several algorithms for searching in this section. We will study the number of steps used by each 
of these algorithms in Section 3.3. 

The general searching problem can be described as follows: Locate an element x in a list of 
distinct elements aj, dy, ..., a,,, or determine that it is not in the list. The solution to this search 
problem is the location of the term in the list that equals x (that is, i is the solution if x = a;) and 
is O if x is not in the list. 


THE LINEAR SEARCH The first algorithm that we will present is called the linear search, 
or sequential search, algorithm. The linear search algorithm begins by comparing x and a}. 
When x = q}, the solution is the location of a,;, namely, 1. When x # a,, compare x with a). If 
x = dy, the solution is the location of a), namely, 2. When x # a), compare x with a}. Continue 
this process, comparing x successively with each term of the list until a match is found, where the 
solution is the location of that term, unless no match occurs. If the entire list has been searched 
without locating x, the solution is 0. The pseudocode for the linear search algorithm is displayed 
as Algorithm 2. 


ALGORITHM 2 The Linear Search Algorithm. 


procedure linear search(x: integer, a), ay, ...,a,: distinct integers) 
i= 1 
while (i < n and x £ a,) 
i:=i+1 
if i < n then location := i 
else location := 0 
return location{location is the subscript of the term that equals x, or is 0 if x is not found} 


Links » 
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THE BINARY SEARCH We will now consider another searching algorithm. This algorithm 
can be used when the list has terms occurring in order of increasing size (for instance: if the 
terms are numbers, they are listed from smallest to largest; if they are words, they are listed 
in lexicographic, or alphabetic, order). This second searching algorithm is called the binary 
search algorithm. It proceeds by comparing the element to be located to the middle term 
of the list. The list is then split into two smaller sublists of the same size, or where one of 
these smaller lists has one fewer term than the other. The search continues by restricting the 
search to the appropriate sublist based on the comparison of the element to be located and 
the middle term. In Section 3.3, it will be shown that the binary search algorithm is much 
more efficient than the linear search algorithm. Example 3 demonstrates how a binary search 
works. 


To search for 19 in the list 
1235678 10 12 13 15 16 18 19 20 22, 

first split this list, which has 16 terms, into two smaller lists with eight terms each, namely, 
1235678 10 12 13 15 16 18 19 20 22. 


Then, compare 19 and the largest term in the first list. Because 10 < 19, the search for 19 can 
be restricted to the list containing the 9th through the 16th terms of the original list. Next, split 
this list, which has eight terms, into the two smaller lists of four terms each, namely, 


12 13 15 16 18 19 20 22. 


Because 16 < 19 (comparing 19 with the largest term of the first list) the search is restricted to 
the second of these lists, which contains the 13th through the 16th terms of the original list. The 
list 18 19 20 22 is split into two lists, namely, 


18 19 20 22. 


Because 19 is not greater than the largest term of the first of these two lists, which is also 19, the 
search is restricted to the first list: 18 19, which contains the 13th and 14th terms of the original 
list. Next, this list of two terms is split into two lists of one term each: 18 and 19. Because 
18 < 19, the search is restricted to the second list: the list containing the 14th term of the list, 
which is 19. Now that the search has been narrowed down to one term, a comparison is made, 
and 19 is located as the 14th term in the original list. < 


We now specify the steps of the binary search algorithm. To search for the integer x in 
the list a,, dy, ...,a,, Where a, < a, < +++ <a,, begin by comparing x with the middle term a, 
of the list, where m = [(n + 1)/2]. (Recall that |x] is the greatest integer not exceeding x.) If 
X > a,,, the search for x is restricted to the second half of the list, which is 4,41, Gna +++ 4 


2 n’ 
If x is not greater than a,,, the search for x is restricted to the first half of the list, which is 
Gy, >, ...,4 


The search has now been restricted to a list with no more than [1/2] elements. (Recall that 
[x] is the smallest integer greater than or equal to x.) Using the same procedure, compare x to 
the middle term of the restricted list. Then restrict the search to the first or second half of the 
list. Repeat this process until a list with one term is obtained. Then determine whether this term 
is x. Pseudocode for the binary search algorithm is displayed as Algorithm 3. 
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ALGORITHM 3 The Binary Search Algorithm. 


procedure binary search (x: integer, a,, a, ..., a„: increasing integers) 
i := 1{i is left endpoint of search interval} 
j := n {jis right endpoint of search interval } 
while i <j 
m:= [(i+/)/2| 
if x > a,, theni:=m-+ 1 
else j := m 
if x = a; then location := i 
else location := 0 
return Jocation{ location is the subscript i of the term a; equal to x, or 0 if x is not found} 


Algorithm 3 proceeds by successively narrowing down the part of the sequence being 
searched. At any given stage only the terms from a; to a; are under consideration. In other 
words, i and j are the smallest and largest subscripts of the remaining terms, respectively. Al- 
gorithm 3 continues narrowing the part of the sequence being searched until only one term 
of the sequence remains. When this is done, a comparison is made to see whether this term 
equals x. 


3.1.3 Sorting 


Ordering the elements of a list is a problem that occurs in many contexts. For example, to 
produce a telephone directory it is necessary to alphabetize the names of subscribers. Similarly, 
producing a directory of songs available for downloading requires that their titles be put in 
alphabetic order. Putting addresses in order in an e-mail mailing list can determine whether there 
are duplicated addresses. Creating a useful dictionary requires that words be put in alphabetical 
order. Similarly, generating a parts list requires that we order them according to increasing part 
number. 

Suppose that we have a list of elements of a set. Furthermore, suppose that we have a way to 
order elements of the set. (The notion of ordering elements of sets will be discussed in detail in 
Section 9.6.) Sorting is putting these elements into a list in which the elements are in increasing 
order. For instance, sorting the list 7, 2, 1, 4, 5, 9 produces the list 1, 2, 4, 5, 7, 9. Sorting the 
list d, h, c, a, f (using alphabetical order) produces the list a, c, d, f, h. 

An amazingly large percentage of computing resources is devoted to sorting one thing or 
another. Hence, much effort has been devoted to the development of sorting algorithms. A 
surprisingly large number of sorting algorithms have been devised using distinct strategies, 
with new ones introduced regularly. In the third volume of his fundamental work The Art of 
Computer Programming [Kn98], Donald Knuth devotes close to 400 pages to sorting, cover- 
ing around 15 different sorting algorithms in depth! More than 100 sorting algorithms have 
been devised, and it is surprising how often new sorting algorithms are developed. Among the 
newest sorting algorithms that have caught on is a widely useful algorithm called Timsort, which 
was invented in 2002, and the library sort, also known as the gapped insertion sort, invented 
in 2006. 

There are many reasons why sorting algorithms interest computer scientists and mathemati- 
cians. Among these reasons are that some algorithms are easier to implement, some algorithms 
are more efficient (either in general, or when given input with certain characteristics, such as 
lists slightly out of order), some algorithms take advantage of particular computer architec- 
tures, and some algorithms are particularly clever. In this section we will introduce two sorting 
algorithms, the bubble sort and the insertion sort. Two other sorting algorithms, the selection 
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sort and the binary insertion sort, are introduced in the exercises, and the shaker sort is in- 
troduced in the Supplementary Exercises. In Section 5.4 we will discuss the merge sort and 
introduce the quick sort in the exercises in that section; the tournament sort is introduced 
in the exercise set in Section 11.2. We cover sorting algorithms both because sorting is an 
important problem and because these algorithms can serve as examples for many important 
concepts. 


THE BUBBLE SORT The bubble sort is one of the simplest sorting algorithms, but not one 
of the most efficient. It puts a list into increasing order by successively comparing adjacent 
elements, interchanging them if they are in the wrong order. To carry out the bubble sort, we 
perform the basic operation, that is, interchanging a larger element with a smaller one following 
it, starting at the beginning of the list, for a full pass. We iterate this procedure until the sort is 
complete. Pseudocode for the bubble sort is given as Algorithm 4. We can imagine the elements 
in the list placed in a column. In the bubble sort, the smaller elements “bubble” to the top 
as they are interchanged with larger elements. The larger elements “sink” to the bottom. This 
is illustrated in Example 4. 


Use the bubble sort to put 3, 2, 4, 1, 5 into increasing order. 


Solution: The steps of this algorithm are illustrated in Figure 1. Begin by comparing the first two 
elements, 3 and 2. Because 3 > 2, interchange 3 and 2, producing the list 2, 3, 4, 1, 5. Because 
3 < 4, continue by comparing 4 and 1. Because 4 > 1, interchange 1 and 4, producing the list 
2, 3, 1,4, 5. Because 4 < 5, the first pass is complete. The first pass guarantees that the largest 
element, 5, is in the correct position. 

The second pass begins by comparing 2 and 3. Because these are in the correct order, 3 and 
1 are compared. Because 3 > 1, these numbers are interchanged, producing 2, 1, 3, 4, 5. Because 
3 < 4, these numbers are in the correct order. It is not necessary to do any more comparisons 
for this pass because 5 is already in the correct position. The second pass guarantees that the 
two largest elements, 4 and 5, are in their correct positions. 

The third pass begins by comparing 2 and 1. These are interchanged because 2 > 1, produc- 
ing 1, 2,3, 4,5. Because 2 < 3, these two elements are in the correct order. It is not necessary 
to do any more comparisons for this pass because 4 and 5 are already in the correct posi- 
tions. The third pass guarantees that the three largest elements, 3, 4, and 5, are in their correct 


positions. 
The fourth pass consists of one comparison, namely, the comparison of 1 and 2. Because 

1 < 2, these elements are in the correct order. This completes the bubble sort. < 
First pass E 2 2 2 Second pass ( 2 2 2 

2 ( 3 3 3 3 4 3 1 

4 4 © 1 1 1 3 

1 1 1 j 4 4 G 

5 5 5 5 5 5 [5] 
Third pass 72 1 Fourth pass 1 eee 

Ç 1 ( 2 ka C : an interchange 

3 3 2 

4 4 4 C : pair in correct order 

5 5 5 numbers in color 


guaranteed to be in correct order 


FIGURE 1 The steps of a bubble sort. 


208 3/ Algorithms 


Links » 


EXAMPLE 5 


ALGORITHM 4 The Bubble Sort. 


procedure bubblesort(a}, ..., a 
for i := l ton— 1 
for j := l ton —i 
if a; > a,,, then interchange a; and a,,, 
{a,, ..., a, is in increasing order} 


n: Teal numbers with n > 2) 


THE INSERTION SORT The insertion sort is a simple sorting algorithm, but it is usually 
not the most efficient. To sort a list with n elements, the insertion sort begins with the second 
element. The insertion sort compares this second element with the first element and inserts it 
before the first element if it does not exceed the first element and after the first element if it 
exceeds the first element. At this point, the first two elements are in the correct order. The third 
element is then compared with the first element, and if it is larger than the first element, it is 
compared with the second element; it is inserted into the correct position among the first three 
elements. 

In general, in the jth step of the insertion sort, the jth element of the list is inserted into 
the correct position in the list of the previously sorted j — 1 elements. To insert the jth element 
in the list, a linear search technique is used (see Exercise 45); the jth element is successively 
compared with the already sorted j — 1 elements at the start of the list until the first element 
that is not less than this element is found or until it has been compared with all j — 1 elements; 
the jth element is inserted in the correct position so that the first j elements are sorted. The 
algorithm continues until the last element is placed in the correct position relative to the al- 
ready sorted list of the first n — 1 elements. The insertion sort is described in pseudocode in 
Algorithm 5. 


Use the insertion sort to put the elements of the list 3, 2, 4, 1, 5 in increasing order. 


Solution: The insertion sort first compares 2 and 3. Because 3 > 2, it places 2 in the first position, 
producing the list 2, 3, 4, 1, 5 (the sorted part of the list is shown in color). At this point, 2 and 
3 are in the correct order. Next, it inserts the third element, 4, into the already sorted part of the 
list by making the comparisons 4 > 2 and 4 > 3. Because 4 > 3, 4 remains in the third position. 
At this point, the list is 2, 3, 4, 1, 5 and we know that the ordering of the first three elements 
is correct. Next, we find the correct place for the fourth element, 1, among the already sorted 
elements, 2, 3,4. Because 1 < 2, we obtain the list 1, 2, 3, 4, 5. Finally, we insert 5 into the 
correct position by successively comparing it to 1, 2, 3, and 4. Because 5 > 4, it stays at the end 
of the list, producing the correct order for the entire list. < 


ALGORITHM 5 The Insertion Sort. 
procedure insertion sort(a,, dy, ...,a,,: real numbers with n > 2) 
for j:=2ton 
i:=1 
while a; > a; 
i:=i+1 
m i= aj; 
for k :=Otoj—i-1 
ajk = Ajk- 
a;:=m 
{a,, ..., qa, is in increasing order} 
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3.1.4 String Matching 


Although searching and sorting are the most commonly encountered problems in computer 
science, many other problems arise frequently. One of these problems asks where a particular 
string of characters P, called the pattern, occurs, if it does, within another string 7, called the 
text. For instance, we can ask whether the pattern 101 can be found within the string 11001011. 
By inspection we can see that the pattern 101 occurs within the text 11001011 at a shift of four 
characters, because 101 is the string formed by the fifth, sixth, and seventh characters of the 
text. On the other hand, the pattern 111 does not occur within the text 110110001101. 

Finding where a pattern occurs in a text string is called string matching. String matching 
plays an essential role in a wide variety of applications, including text editing, spam filters, sys- 
tems that look for attacks in a computer network, search engines, plagiarism detection, bioinfor- 
matics, and many other important applications. For example, in text editing, the string matching 
problem arises whenever we need to find all occurrences of a string so that we can replace this 
string with a different string. Search engines look for matching of search keywords with words 
on web pages. Many problems in bioinformatics arise in the study of DNA molecules, which are 
made up of four bases: thymine (T), adenine (A), cytosine (C), and guanine (G). The process of 
DNA sequencing is the determination of the order of the four bases in DNA. This leads to string 
matching problems involving strings made up from the four letters T, A, C, and G. For instance, 
we can ask whether the pattern CAG occurs in the text CATCACAGAGA. The answer is yes, 
because it occurs with a shift of five characters. Solving questions about the genome requires 
the use of efficient algorithms for string matching, especially because a string representing a 
human genome is about 3 x 10° characters long. 

We will now describe a brute force algorithm, Algorithm 6, for string matching, called 
the naive string matcher. The input to this algorithm is the pattern we wish to match, P = 
P\P2 ++» Pm and the text, T = tih ... t, When this pattern begins at position s + 1 in the text T, 
we say that P occurs with shift s in T, that is, when t41 = Py, t42 = Po +++ > by = Pm TO find all 
valid shifts, the naive string matcher runs through all possible shifts s from s = 0 to s =n — m, 
checking whether s is a valid shift. In Figure 2, we display the operation of Algorithm 6 when 
it is used to search for the pattern P = eye in the text T = eceyeye. 


ALGORITHM 6 Naive String Matcher. 


procedure string match (n, m: positive integers, m < n, ti, ty, ...5 ty» Py» Pa» -++» Pm: Characters) 
for s := Oton — m 
j:=1 
while (j < m and ¢,,; = p;) 
j:=j+1 


if j > m then print “s is a valid shift” 


elclelylelyle elclely 
s=3 j s=4 
Je |yle c 


FIGURE 2 The steps of the naive string matcher with P = eye in T = eceyeye. Matches 
are identified with a solid line and mismatches with a jagged line. The algorithm finds 
two valid shifts, s = 2 and s = 4. 
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Many other string matching algorithms have been developed besides the naive string 
matcher. These algorithms use a surprisingly wide variety of approaches to make them more effi- 
cient than the naive string matcher. To learn more about these algorithms, consult [CoLeRiSt09], 
as well as books on algorithms in bioinformatics. 


3.1.5 Greedy Algorithms 


Many algorithms we will study in this book are designed to solve optimization problems. 
The goal of such problems is to find a solution to the given problem that either minimizes 
or maximizes the value of some parameter. Optimization problems studied later in this text 
include finding a route between two cities with least total mileage, determining a way to encode 
messages using the fewest bits possible, and finding a set of fiber links between network nodes 
using the least amount of fiber. 

Surprisingly, one of the simplest approaches often leads to a solution of an optimization 
problem. This approach selects the best choice at each step, instead of considering all sequences 
of steps that may lead to an optimal solution. Algorithms that make what seems to be the “best” 
choice at each step are called greedy algorithms. Once we know that a greedy algorithm finds 
a feasible solution, we need to determine whether it has found an optimal solution. (Note that 
we call the algorithm “greedy” whether or not it finds an optimal solution.) To do this, we 
either prove that the solution is optimal or we show that there is a counterexample where the 
algorithm yields a nonoptimal solution. To make these concepts more concrete, we will consider 
the cashier’s algorithm that makes change using coins. (This algorithm is called the cashier’s 
algorithm because cashiers often used this algorithm for making change in the days before cash 
registers became electronic.) 


Consider the problem of making n cents change with quarters, dimes, nickels, and pennies, and 
using the least total number of coins. We can devise a greedy algorithm for making change for 
n cents by making a locally optimal choice at each step; that is, at each step we choose the coin 
of the largest denomination possible to add to the pile of change without exceeding n cents. For 
example, to make change for 67 cents, we first select a quarter (leaving 42 cents). We next select 
a second quarter (leaving 17 cents), followed by a dime (leaving 7 cents), followed by a nickel 
(leaving 2 cents), followed by a penny (leaving 1 cent), followed by a penny. < 


We display the cashier’s algorithm for cents, using any set of denominations of coins, as 
Algorithm 7. 


ALGORITHM 7 Cashier’s Algorithm. 


procedure change(c,, cy, ..., c,: values of denominations of coins, where 
Ci > Cy > + >C, n: a positive integer) 
for i := 1 tor 
d; := 0 {d, counts the coins of denomination c; used} 
while n > c; 
d; := d; + 1 {add a coin of denomination c;} 
n:=n-— C; 
{d; is the number of coins of denomination c; in the change for i = 1, 2, ..., r} 


We have described the cashier’s algorithm, a greedy algorithm for making change, using 
any finite set of coins with denominations c}, C3, ...,c,. In the particular case where the four 
denominations are quarters, dimes, nickels, and pennies, we have c} = 25, c, = 10, c} = 5, and 
c4 = 1. For this case, we will show that this algorithm leads to an optimal solution in the sense 
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that it uses the fewest coins possible. Before we embark on our proof, we show that there are sets 
of coins for which the cashier’s algorithm (Algorithm 7) does not necessarily produce change 
using the fewest coins possible. For example, if we have only quarters, dimes, and pennies (and 
no nickels) to use, the cashier’s algorithm would make change for 30 cents using six coins—a 
quarter and five pennies—whereas we could have used three coins, namely, three dimes. 


If n is a positive integer, then n cents in change using quarters, dimes, nickels, and pennies 
using the fewest coins possible has at most two dimes, at most one nickel, at most four pen- 
nies, and cannot have two dimes and a nickel. The amount of change in dimes, nickels, and 
pennies cannot exceed 24 cents. 


Proof: We use a proof by contradiction. We will show that if we had more than the specified 
number of coins of each type, we could replace them using fewer coins that have the same value. 
We note that if we had three dimes we could replace them with a quarter and a nickel, if we 
had two nickels we could replace them with a dime, if we had five pennies we could replace 
them with a nickel, and if we had two dimes and a nickel we could replace them with a quarter. 
Because we can have at most two dimes, one nickel, and four pennies, but we cannot have two 
dimes and a nickel, it follows that 24 cents is the most money we can have in dimes, nickels, 
and pennies when we make change using the fewest number of coins for n cents. < 


The cashier’s algorithm (Algorithm 7) always makes changes using the fewest coins possible 
when change is made from quarters, dimes, nickels, and pennies. 


Proof: We will use a proof by contradiction. Suppose that there is a positive integer n such that 
there is a way to make change for n cents using quarters, dimes, nickels, and pennies that uses 
fewer coins than the greedy algorithm finds. We first note that q’, the number of quarters used 
in this optimal way to make change for n cents, must be the same as q, the number of quarters 
used by the greedy algorithm. To show this, first note that the greedy algorithm uses the most 
quarters possible, so g’ < q. However, it is also the case that g’ cannot be less than q. If it were, 
we would need to make up at least 25 cents from dimes, nickels, and pennies in this optimal 
way to make change. But this is impossible by Lemma 1. 

Because there must be the same number of quarters in the two ways to make change, the 
value of the dimes, nickels, and pennies in these two ways must be the same, and these coins 
are worth no more than 24 cents. There must be the same number of dimes, because the greedy 
algorithm used the most dimes possible and by Lemma 1, when change is made using the fewest 
coins possible, at most one nickel and at most four pennies are used, so that the most dimes 
possible are also used in the optimal way to make change. Similarly, we have the same number 
of nickels and, finally, the same number of pennies. < 


A greedy algorithm makes the best choice at each step according to a specified criterion. 
The next example shows that it can be difficult to determine which of many possible criteria to 
choose. 


Suppose we have a group of proposed talks with preset start and end times. Devise a greedy 
algorithm to schedule as many of these talks as possible in a lecture hall, under the assumptions 
that once a talk starts, it continues until it ends, no two talks can proceed at the same time, and 
a talk can begin at the same time another one ends. Assume that talk j begins at time s, (where 
s stands for start) and ends at time e; (where e stands for end). 


Solution: To use a greedy algorithm to schedule the most talks, that is, an optimal schedule, we 
need to decide how to choose which talk to add at each step. There are many criteria we could 
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use to select a talk at each step, where we chose from the talks that do not overlap talks already 
selected. For example, we could add talks in order of earliest start time, we could add talks in 
order of shortest time, we could add talks in order of earliest finish time, or we could use some 
other criterion. 

We now consider these possible criteria. Suppose we add the talk that starts earliest 
among the talks compatible with those already selected. We can construct a counterexample 
to see that the resulting algorithm does not always produce an optimal schedule. For instance, 
suppose that we have three talks: Talk 1 starts at 8 A.M. and ends at 12 noon, Talk 2 starts at 9 
A.M. and ends at 10 A.M., and Talk 3 starts at 11 A.M. and ends at 12 noon. We first select the 
Talk 1 because it starts earliest. But once we have selected Talk 1 we cannot select either Talk 2 
or Talk 3 because both overlap Talk 1. Hence, this greedy algorithm selects only one talk. This 
is not optimal because we could schedule Talk 2 and Talk 3, which do not overlap. 

Now suppose we add the talk that is shortest among the talks that do not overlap any of those 
already selected. Again we can construct a counterexample to show that this greedy algorithm 
does not always produce an optimal schedule. So, suppose that we have three talks: Talk 1 starts 
at 8 A.M. and ends at 9:15 A.M., Talk 2 starts at 9 A.M. and ends at 10 A.M., and Talk 3 starts at 
9:45 A.M. and ends at 11 A.M. We select Talk 2 because it is shortest, requiring one hour. Once 
we select Talk 2, we cannot select either Talk 1 or Talk 3 because neither is compatible with 
Talk 2. Hence, this greedy algorithm selects only one talk. However, it is possible to select two 
talks, Talk 1 and Talk 3, which are compatible. 

However, it can be shown that we schedule the most talks possible if in each step we 
select the talk with the earliest ending time among the talks compatible with those already 
selected. We will prove this in Chapter 5 using the method of mathematical induction. The 
first step we will make is to sort the talks according to increasing finish time. After this sort- 
ing, we relabel the talks so that e; < e, < +: < e,. The resulting greedy algorithm is given as 


Algorithm 8. < 


ALGORITHM 8 Greedy Algorithm for Scheduling Talks. 


procedure schedule(s; < sy < = < s„: start times of talks, 
e; Se, < = <e,: ending times of talks) 
sort talks by finish time and reorder so that e} < e, < + < en 
S:=9 
for j:= 1ton 
if talk j is compatible with S then 
S:= SU {talk j} 
return S{S is the set of talks scheduled} 


3.1.6 The Halting Problem 


We will now describe a proof of one of the most famous theorems in computer science. We 
will show that there is a problem that cannot be solved using any procedure. That is, we will 
show there are unsolvable problems. The problem we will study is the halting problem. It 
asks whether there is a procedure that does this: It takes as input a computer program and 
input to the program and determines whether the program will eventually stop when run with 
this input. It would be convenient to have such a procedure, if it existed. Certainly being able 
to test whether a program entered into an infinite loop would be helpful when writing and 
debugging programs. However, in 1936 Alan Turing showed that no such procedure exists (see 
his biography in Section 13.4). 

Before we present a proof that the halting problem is unsolvable, first note that we cannot 
simply run a program and observe what it does to determine whether it terminates when run 
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If H(P, P) = “halts,” 


P as program P P then loop forever 
g rogram rogram = |__» 
Input HED) Output K(P) 
Program P H(P, P) 
a 
|________» 
P as input If H(P, P) = “loops forever,” 
then halt 


FIGURE 3 Showing that the halting problem is unsolvable. 


with the given input. If the program halts, we have our answer, but if it is still running after any 
fixed length of time has elapsed, we do not know whether it will never halt or we just did not 
wait long enough for it to terminate. After all, it is not hard to design a program that will stop 
only after more than a billion years has elapsed. 

We will describe Turing’s proof that the halting problem is unsolvable; it is a proof by 
contradiction. (The reader should note that our proof is not completely rigorous, because we 
have not explicitly defined what a procedure is. To remedy this, the concept of a Turing machine 
is needed. This concept is introduced in Section 13.5.) 


Proof: Assume there is a solution to the halting problem, a procedure called H(P I). The pro- 

È cedure H(P, I) takes two inputs, one a program P and the other 7, an input to the program P. 
H(P,I) generates the string “halt” as output if H determines that P stops when given 7 as in- 
put. Otherwise, H(P I) generates the string “loops forever” as output. We will now derive a 
contradiction. 

When a procedure is coded, it is expressed as a string of characters; this string can be 
interpreted as a sequence of bits. This means that a program itself can be used as data. Therefore, 
a program can be thought of as input to another program, or even itself. Hence, H can take a 
program P as both of its inputs, which are a program and input to this program. H should be 
able to determine whether P will halt when it is given a copy of itself as input. 

To show that no procedure H exists that solves the halting problem, we construct a simple 
procedure K(P), which works as follows, making use of the output H(P P). If the output of 
H(P, P) is “loops forever,’ which means that P loops forever when given a copy of itself as 
input, then K(P) halts. If the output of H(P, P) is “halt,;’ which means that P halts when given 
a copy of itself as input, then K(P) loops forever. That is, K(P) does the opposite of what the 
output of H(P P) specifies. (See Figure 3.) 

Now suppose we provide K as intput to K. We note that if the output of H(K, K) is “loops 
forever,” then by the definition of K, we see that K(K) halts. This means that by the definition of 
H, the output of H(K, K) is “halt,” which is a contradiction. Otherwise, if the output of H(K, K) 
is “halts,” then by the definition of K, we see that K(K) loops forever, which means that by 
the definition of H, the output of H(K, K) is “loops forever.” This is also a contradiction. Thus, 
H cannot always give the correct answers. Consequently, there is no procedure that solves the 


halting problem. < 
Exercises 
1. List all the steps used by Algorithm 1 to find the maxi- b) procedure divide(n: positive integer) 
mum of the list 1, 8, 12, 9, 11, 2, 14, 5, 10, 4. while n > 0 
m:=1/n 
. : e ! n:=n-1 
2. Determine which characteristics of an algorithm de- c) procedure sum(n: positive integer) 
scribed in the text (after Algorithm 1) the following pro- sum:=0 
cedures have and which they lack. while i < 10 
a) procedure double(n: positive integer) sum := sum +i 
while n > 0 d) procedure choose(a, b: integers) 


n:=2n x := either a or b 
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Devise an algorithm that finds the sum of all the integers 
in a list. 


. Describe an algorithm that takes as input a list of n in- 


tegers and produces as output the largest difference ob- 
tained by subtracting an integer in the list from the one 
following it. 


Describe an algorithm that takes as input a list of n inte- 
gers in nondecreasing order and produces the list of all 
values that occur more than once. (Recall that a list of 
integers is nondecreasing if each integer in the list is at 
least as large as the previous integer in the list.) 


Describe an algorithm that takes as input a list of n in- 
tegers and finds the number of negative integers in the 
list. 


. Describe an algorithm that takes as input a list of n inte- 


gers and finds the location of the last even integer in the 
list or returns 0 if there are no even integers in the list. 


. Describe an algorithm that takes as input a list of n dis- 


tinct integers and finds the location of the largest even 
integer in the list or returns 0 if there are no even integers 
in the list. 


. A palindrome is a string that reads the same forward 


and backward. Describe an algorithm for determining 
whether a string of n characters is a palindrome. 


Devise an algorithm to compute x”, where x is areal num- 
ber and n is an integer. [Hint: First give a procedure for 
computing x” when n is nonnegative by successive mul- 
tiplication by x, starting with 1. Then extend this proce- 
dure, and use the fact that x™” = 1/x” to compute x” when 
n is negative. ] 


Describe an algorithm that interchanges the values of the 
variables x and y, using only assignments. What is the 
minimum number of assignment statements needed to do 
this? 


Describe an algorithm that uses only assignment state- 
ments that replaces the triple (x, y, z) with (y, z, x). What is 
the minimum number of assignment statements needed? 


List all the steps used to search for 9 in the sequence 1, 
3, 4, 5, 6, 8, 9, 11 using 


a) a linear search. b) a binary search. 


List all the steps used to search for 7 in the sequence given 
in Exercise 13 for both a linear search and a binary search. 


Describe an algorithm that inserts an integer x in the ap- 
propriate position into the list a}, a, ...,a, of integers 
that are in increasing order. 


Describe an algorithm for finding the smallest integer in 
a finite sequence of natural numbers. 


Describe an algorithm that locates the first occurrence of 
the largest element in a finite list of integers, where the 
integers in the list are not necessarily distinct. 


Describe an algorithm that locates the last occurrence of 
the smallest element in a finite list of integers, where the 
integers in the list are not necessarily distinct. 


19. 


20. 


21. 


22. 


23 


24. 


25. 


26. 


27. 


28. 


Describe an algorithm that produces the maximum, me- 
dian, mean, and minimum of a set of three integers. (The 
median of a set of integers is the middle element in the 
list when these integers are listed in order of increasing 
size. The mean of a set of integers is the sum of the inte- 
gers divided by the number of integers in the set.) 


Describe an algorithm for finding both the largest and the 
smallest integers in a finite sequence of integers. 


Describe an algorithm that puts the first three terms of 
a sequence of integers of arbitrary length in increasing 
order. 


Describe an algorithm to find the longest word in an En- 
glish sentence (where a sentence is a sequence of sym- 
bols, either a letter or a blank, which can then be broken 
into alternating words and blanks). 


Describe an algorithm that determines whether a func- 
tion from a finite set of integers to another finite set of 
integers is onto. 


Describe an algorithm that determines whether a func- 
tion from a finite set to another finite set is one-to-one. 


Describe an algorithm that will count the number of 1s 
in a bit string by examining each bit of the string to de- 
termine whether it is a 1 bit. 


Change Algorithm 3 so that the binary search procedure 
compares x to a,, at each stage of the algorithm, with the 
algorithm terminating if x = a,,. What advantage does 
this version of the algorithm have? 


The ternary search algorithm locates an element in a 
list of increasing integers by successively splitting the list 
into three sublists of equal (or as close to equal as pos- 
sible) size, and restricting the search to the appropriate 
piece. Specify the steps of this algorithm. 


Specify the steps of an algorithm that locates an element 
in a list of increasing integers by successively splitting 
the list into four sublists of equal (or as close to equal as 
possible) size, and restricting the search to the appropri- 
ate piece. 


In a list of elements the same element may appear several 
times. A mode of such a list is an element that occurs at least 
as often as each of the other elements; a list has more than 
one mode when more than one element appears the maximum 
number of times. 


29. 


30. 


31. 


Devise an algorithm that finds a mode in a list of nonde- 
creasing integers. (Recall that a list of integers is nonde- 
creasing if each term is at least as large as the preceding 
term.) 


Devise an algorithm that finds all modes. (Recall that a 

list of integers is nondecreasing if each term of the list is 

at least as large as the preceding term.) 

Two strings are anagrams if each can be formed from the 

other string by rearranging its characters. Devise an algo- 

rithm to determine whether two strings are anagrams 

a) by first finding the frequency of each character that 
appears in the strings. 

b) by first sorting the characters in both strings. 
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32. 


33. 


34. 


35. 


36. 


37. 


38. 


40. 


41. 


42. 


.,x,, find the two that are 


n? 


Given n real numbers x}, x», .. 

closest together by 

a) a brute force algorithm that finds the distance between 
every pair of these numbers. 

b) sorting the numbers and computing the least number 
of distances needed to solve the problem. 

Devise an algorithm that finds the first term of a se- 

quence of integers that equals some previous term in the 

sequence. 


Devise an algorithm that finds all terms of a finite se- 
quence of integers that are greater than the sum of all 
previous terms of the sequence. 


Devise an algorithm that finds the first term of a sequence 
of positive integers that is less than the immediately pre- 
ceding term of the sequence. 


Use the bubble sort to sort 6, 2, 3, 1, 5, 4, showing the 
lists obtained at each step. 


Use the bubble sort to sort 3, 1, 5, 7, 4, showing the lists 
obtained at each step. 


Use the bubble sort to sort d, f, k, m, a, b, showing the 
lists obtained at each step. 


Adapt the bubble sort algorithm so that it stops when no 
interchanges are required. Express this more efficient ver- 
sion of the algorithm in pseudocode. 


Use the insertion sort to sort the list in Exercise 36, show- 
ing the lists obtained at each step. 


Use the insertion sort to sort the list in Exercise 37, show- 
ing the lists obtained at each step. 


Use the insertion sort to sort the list in Exercise 38, show- 
ing the lists obtained at each step. 


The selection sort begins by finding the least element in the 
list. This element is moved to the front. Then the least element 
among the remaining elements is found and put into the sec- 
ond position. This procedure is repeated until the entire list 
has been sorted. 


43. 


46. 


47. 


48. 


Sort these lists using the selection sort. 
a) 3,5,4, 1,2 b) 5, 4, 3, 2, 1 
e) 1,.2,3,4,5 


. Write the selection sort algorithm in pseudocode. 


. Describe an algorithm based on the linear search for de- 


termining the correct position in which to insert a new 
element in an already sorted list. 


Describe an algorithm based on the binary search for de- 
termining the correct position in which to insert a new 
element in an already sorted list. 


How many comparisons does the insertion sort use to sort 
the list 1, 2, ..., n? 


How many comparisons does the insertion sort use to sort 
the list n, n — 1, ...,2, 1? 


The binary insertion sort is a variation of the insertion sort 
that uses a binary search technique (see Exercise 46) rather 
than a linear search technique to insert the ith element in the 
correct place among the previously sorted elements. 


49. 


50. 


* 51. 
52. 


53. 


54. 


55. 


56. 


57. 


58. 


59. 


60. 


61. 


62. 
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Show all the steps used by the binary insertion sort to sort 
the list 3, 2, 4, 5, 1, 6. 

Compare the number of comparisons used by the inser- 
tion sort and the binary insertion sort to sort the list 7, 4, 
3, 8, 1,5, 4, 2. 

Express the binary insertion sort in pseudocode. 


a) Devise a variation of the insertion sort that uses a lin- 
ear search technique that inserts the jth element in the 
correct place by first comparing it with the (j — 1)st 
element, then the (j — 2)th element if necessary, and 
so on. 

b) Use your algorithm to sort 3, 2, 4, 5, 1, 6. 

c) Answer Exercise 47 using this algorithm. 

d) Answer Exercise 48 using this algorithm. 


When a list of elements is in close to the correct order, 
would it be better to use an insertion sort or its variation 
described in Exercise 52? 


List all the steps the naive string matcher uses to find all 
occurrences of the pattern FE in the text COVFEFE. 


List all the steps the naive string matcher uses to find all 
occurrences of the pattern ACG in the text TACAGACG. 


Use the cashier’s algorithm to make change using quar- 
ters, dimes, nickels, and pennies for 

a) 87 cents. b) 49 cents. 

c) 99 cents. d) 33 cents. 


Use the cashier’s algorithm to make change using quar- 
ters, dimes, nickels, and pennies for 

a) 51 cents. b) 69 cents. 

c) 76 cents. d) 60 cents. 

Use the cashier’s algorithm to make change using quar- 
ters, dimes, and pennies (but no nickels) for each of 
the amounts given in Exercise 56. For which of these 
amounts does the greedy algorithm use the fewest coins 
of these denominations possible? 


Use the cashier’s algorithm to make change using quar- 
ters, dimes, and pennies (but no nickels) for each of 
the amounts given in Exercise 57. For which of these 
amounts does the greedy algorithm use the fewest coins 
of these denominations possible? 


Show that if there were a coin worth 12 cents, the 
cashier’s algorithm using quarters, 12-cent coins, dimes, 
nickels, and pennies would not always produce change 
using the fewest coins possible. 


Use Algorithm 7 to schedule the largest number of talks 
ina lecture hall from a proposed set of talks, if the starting 
and ending times of the talks are 9:00 A.M. and 9:45 A.M.; 
9:30 A.M. and 10:00 A.M.; 9:50 A.M. and 10:15 A.M.; 
10:00 A.M. and 10:30 A.M.; 10:10 A.M. and 10:25 A.M.; 
10:30 A.M. and 10:55 A.M.; 10:15 A.M. and 10:45 A.M.; 
10:30 A.M. and 11:00 A.M.; 10:45 A.M. and 11:30 A.M.; 
10:55 A.M. and 11:25 A.M.; 11:00 A.M. and 11:15 A.M. 


Show that a greedy algorithm that schedules talks in a 
lecture hall, as described in Example 7, by selecting at 
each step the talk that overlaps the fewest other talks, does 
not always produce an optimal schedule. 
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*63. a) Devise a greedy algorithm that determines the fewest 
lecture halls needed to accommodate n talks given the 
starting and ending time for each talk. 

b) Prove that your algorithm is optimal. 

Suppose we have s men mı, mū, ...,m, and s women 

W1, Wz, --., Wy. We wish to match each person with a mem- 

ber of the opposite gender. Furthermore, suppose that each 

person ranks, in order of preference, with no ties, the people 
of the opposite gender. We say that a matching of people of 
opposite genders to form couples is stable if we cannot find 

a man m and a woman w who are not assigned to each other 

such that m prefers w over his assigned partner and w prefers 

m to her assigned partner. 

64. Suppose we have three men m,, m,, and m, and three 
women w4, w3, and w3. Furthermore, suppose that the 
preference rankings of the men for the three women, from 
highest to lowest, are m4: W3, W1, W2; M3: W1, Wz, W3; M3: 
Wz, W3, w1; and the preference rankings of the women for 
the three men, from highest to lowest, are w4: m,, my, M3; 
w: My, M4, M3; W3: M3, My, M; . For each of the six possi- 
ble matchings of men and women to form three couples, 
determine whether this matching is stable. 


The deferred acceptance algorithm, also known as the 
Gale-Shapley algorithm, can be used to construct a stable 
matching of men and women. In this algorithm, members of 
one gender are the suitors and members of the other gender 
the suitees. The algorithm uses a sequence of rounds; in each 
round every suitor whose proposal was rejected in the previ- 
ous round proposes to his or her highest ranking suitee who 
has not already rejected a proposal from this suitor. A suitee 
rejects all proposals except that from the suitor that this suitee 
ranks highest among all the suitors who have proposed to this 
suitee in this round or previous rounds. The proposal of this 
highest ranking suitor remains pending and is rejected in a 
later round if a more appealing suitor proposes in that round. 
The series of rounds ends when every suitor has exactly one 
pending proposal. All pending proposals are then accepted. 
65. Write the deferred acceptance algorithm in pseudocode. 
66. Show that the deferred acceptance algorithm terminates. 

*67. Show that the deferred acceptance always terminates 

with a stable assignment. 


The Growth of Functions 


An element of a sequence is called a majority element if 
it occurs repeatedly for more than half the terms of the se- 
quence. The Boyer-Moore majority vote algorithm (named 
after Robert Boyer and J. Strother Moore) finds the majority 
element of a sequence, if it exists. The algorithm maintains 
a counter that is initially set to 0 and a temporary candidate 
for a majority element initially with no assigned value. The 
algorithm processes the elements in order. When it processes 
the first element, this element becomes the majority candi- 
date and the counter is set equal to 1. Then, as it processes the 
remaining elements in order, if the counter is 0, this element 
becomes the majority candidate and the counter is set equal to 
1, while if the counter is nonzero, the counter is incremented 
(that is, 1 is added to it) or decremented (that is, 1 is subtracted 
from it), depending on whether this element equals the cur- 
rent candidate. After all the terms are processed, the majority 
candidate is the majority element, if it exists. 


68. a) Explain why a sequence has at most one majority 
element. 


b) Show all the steps of the Boyer-Moore majority vote 
algorithm when given the sequence 2, 1, 3, 3, 2, 3. 


c) Express the Boyer-Moore majority vote algorithm in 
pseudocode. 


d) Explain how you can determine whether the major- 
ity candidate element produced by the Boyer-Moore 
algorithm is actually a majority element. 


*69. a) Prove that the Boyer-Moore majority vote algorithm 
outputs the majority element of a sequence, if it exists. 


b) Prove or disprove that the majority candidate of the 
Boyer-Moore majority vote algorithm will be a mode 
of the sequence (that is, its most common element) 
even when no majority element exists. 


70. Show that the problem of determining whether a program 
with a given input ever prints the digit 1 is unsolvable. 


71. Show that the following problem is solvable. Given two 
programs with their inputs and the knowledge that ex- 
actly one of them halts, determine which halts. 


72. Show that the problem of deciding whether a specific pro- 
gram with a specific input halts is solvable. 


3.2.1 Introduction 


In Section 3.1 we discussed the concept of an algorithm. We introduced algorithms that solve a 
variety of problems, including searching for an element in a list and sorting a list. In Section 3.3 
we will study the number of operations used by these algorithms. In particular, we will estimate 
the number of comparisons used by the linear and binary search algorithms to find an element 
in a sequence of n elements. We will also estimate the number of comparisons used by the 
bubble sort and by the insertion sort to sort a list of n elements. The time required to solve a 
problem depends on more than only the number of operations it uses. The time also depends on 
the hardware and software used to run the program that implements the algorithm. However, 
when we change the hardware and software used to implement an algorithm, we can closely 
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approximate the time required to solve a problem of size n by multiplying the previous time 
required by a constant. For example, on a supercomputer we might be able to solve a problem 
of size n a million times faster than we can on a PC. However, this factor of one million will 
not depend on n (except perhaps in some minor ways). One of the advantages of using big-O 
notation, which we introduce in this section, is that we can estimate the growth of a function 
without worrying about constant multipliers or smaller order terms. This means that, using big- 
O notation, we do not have to worry about the hardware and software used to implement an 
algorithm. Furthermore, using big-O notation, we can assume that the different operations used 
in an algorithm take the same time, which simplifies the analysis considerably. 

Big-O notation is used extensively to estimate the number of operations an algorithm uses 
as its input grows. With the help of this notation, we can determine whether it is practical to 
use a particular algorithm to solve a problem as the size of the input increases. Furthermore, 
using big-O notation, we can compare two algorithms to determine which is more efficient as 
the size of the input grows. For instance, if we have two algorithms for solving a problem, one 
using 1007? + 17n + 4 operations and the other using n° operations, big-O notation can help us 
see that the first algorithm uses far fewer operations when n is large, even though it uses more 
operations for small values of n, such as n = 10. 

This section introduces big-O notation and the related big-Omega and big-Theta notations. 
We will explain how big-O big-Omega, and big-Theta estimates are constructed and establish 
estimates for some important functions that are used in the analysis of algorithms. 


3.2.2 Big-O Notation 


The growth of functions is often described using a special notation. Definition 1 describes this 
notation. 


Let f and g be functions from the set of integers or the set of real numbers to the set of real 
numbers. We say that f(x) is O(g(x)) if there are constants C and k such that 


Fœ < Cle@)| 


whenever x > k. [This is read as “f (x) is big-oh of e(x).”] 


Remark: Intuitively, the definition that f(x) is O(g(x)) says that f(x) grows slower than some 
fixed multiple of g(x) as x grows without bound. 


The constants C and k in the definition of big-O notation are called witnesses to the rela- 
tionship f(x) is O(g(x)). To establish that f(x) is O(g(x)) we need only one pair of witnesses to 
this relationship. That is, to show that f(x) is O(g(x)), we need find only one pair of constants C 
and k, the witnesses, such that |f(x)| < C|g(x)| whenever x > k. 

Note that when there is one pair of witnesses to the relationship f(x) is O(g(x)), there are 
infinitely many pairs of witnesses. To see this, note that if C and k are one pair of witnesses, 
then any pair C’ and k’, where C < C’ and k < Kk’, is also a pair of witnesses, because [f(x)| < 
Clg(x)| < C’|g()| whenever x > k’ > k. 


THE HISTORY OF BIG-O NOTATION Big-O notation has been used in mathematics for 
more than a century. In computer science it is widely used in the analysis of algorithms, as 
will be seen in Section 3.3. The German mathematician Paul Bachmann first introduced big-O 
notation in 1892 in an important book on number theory. The big-O symbol is sometimes called 
a Landau symbol after the German mathematician Edmund Landau, who used this notation 
throughout his work. The use of big-O notation in computer science was popularized by Donald 
Knuth, who also introduced the big-Q and big-© notations defined later in this section. 
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WORKING WITH THE DEFINITION OF BIG-O NOTATION A useful approach for find- 
ing a pair of witnesses is to first select a value of k for which the size of [f(x)| can be readily 
estimated when x > k and to see whether we can use this estimate to find a value of C for which 
If@)| < Clg(x)| for x > k. This approach is illustrated in Example 1. 


Show that f(x) = x? + 2x + 1 is O(x’). 


Solution: We observe that we can readily estimate the size of f(x) when x > 1 because x < x? 
and 1 < x? when x > 1. It follows that 


0<x 42x4+1<2° 42° 42 =42 


whenever x > 1, as shown in Figure 1. Consequently, we can take C = 4 and k = 1 as witnesses 
to show that f(x) is O(x7). That is, f(x) = x? + 2x + 1 < 4x? whenever x > 1. (Note that it is not 
necessary to use absolute values here because all functions in these equalities are positive when 
x is positive.) 

Alternatively, we can estimate the size of f(x) when x > 2. When x > 2, we have 2x < x 
and 1 < x. Consequently, if x > 2, we have 


0<x*4+2x4+1<2° 42° 427 = 3x’. 


It follows that C = 3 and k = 2 are also witnesses to the relation f(x) is O(x). 

Observe that in the relationship “f(x) is O(x),” x? can be replaced by any function that has 
larger values than x? for all x > k for some positive real number k. For example, f(x) is O(x°), 
f(x) is OG? +x +7), and so on. 

It is also true that x? is O(x? + 2x + 1), because x? < x? +2x+1 whenever x > 1. This 
means that C = 1 and k = 1 are witnesses to the relationship x? is OQ? + 2x + 1). < 


Note that in Example 1 we have two functions, f(x) = x? +2x + 1 and g(x) = x*, such 
that f(x) is O(g(x)) and g(x) is O(f(x))—the latter fact following from the inequality 
x? < x? +2x+ 1, which holds for all nonnegative real numbers x. We say that two functions 


Ax?) px? 42x41 x 


The part of the graph of f(x) =x? + 2x + 1 
that satisfies f(x) < 4x? is shown in color. 


x? 42x41< 4x? forx> 1 


| | 
1 2 


FIGURE 1 The function x? + 2x + 1 is O(x?). 


Source: Bachmann, Paul. 
Die Arithmetik Der 
Quadratischen Formen. 
Verlag und Druck von BG 
Teubner. Leipzig. Berlin. 


1923 
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f(x) and g(x) that satisfy both of these big-O relationships are of the same order. We will 
return to this notion later in this section. 


Remark: The fact that f(x) is O(g(x)) is sometimes written f(x) = O(g(x)). However, the equals 
sign in this notation does not represent a genuine equality. Rather, this notation tells us that an 
inequality holds relating the values of the functions f and g for sufficiently large numbers in the 
domains of these functions. However, it is acceptable to write f(x) € O(g(x)) because O(g(x)) 
represents the set of functions that are O(g(x)). 


When f(x) is O(g(x)), and h(x) is a function that has larger absolute values than g(x) does 
for sufficiently large values of x, it follows that f(x) is O(h(x)). In other words, the function g(x) 
in the relationship f(x) is O(g(x)) can be replaced by a function with larger absolute values. To 
see this, note that if 


IFI < Cle@| if x >k, 
and if |A(x)| > |e(x)| for all x > k, then 
[f@|<Clh@| ifx>k. 


Hence, f(x) is O(h(x)). 

When big-O notation is used, the function g in the relationship f(x) is O(g(x)) is often chosen 
to have the smallest growth rate of the functions belonging to a set of reference functions, such 
as functions of the form x”, where n is a positive real number. (Important reference functions 
are discussed later in this section.) 

In subsequent discussions, we will almost always deal with functions that take on only 
positive values. All references to absolute values can be dropped when working with big-O 
estimates for such functions. Figure 2 illustrates the relationship f(x) is O(g(x)). 


PAUL GUSTAV HEINRICH BACHMANN (1837-1920) Paul Bachmann, the son of a Lutheran pastor, 
shared his father’s pious lifestyle and love of music. His mathematical talent was discovered by one of his teach- 
ers, even though he had difficulties with some of his early mathematical studies. After recuperating from tuber- 
culosis in Switzerland, Bachmann studied mathematics, first at the University of Berlin and later at Gottingen, 
where he attended lectures presented by the famous number theorist Dirichlet. He received his doctorate under 
the German number theorist Kummer in 1862; his thesis was on group theory. Bachmann was a professor at 
Breslau and later at Miinster. After he retired from his professorship, he continued his mathematical writing, 
played the piano, and served as a music critic for newspapers. Bachmann’s mathematical writings include a 
five-volume survey of results and methods in number theory, a two-volume work on elementary number the- 
ory, a book on irrational numbers, and a book on the famous conjecture known as Fermat’s Last Theorem. He 
introduced big-O notation in his 1892 book Analytische Zahlentheorie. 


EDMUND LANDAU (1877-1938) Edmund Landau, the son of a Berlin gynecologist, attended high school 
and university in Berlin. He received his doctorate in 1899, under the direction of Frobenius. Landau first 
taught at the University of Berlin and then moved to Gottingen, where he was a full professor until the Nazis 
forced him to stop teaching. Landau’s main contributions to mathematics were in the field of analytic number 
theory. In particular, he established several important results concerning the distribution of primes. He au- 
thored a three-volume exposition on number theory as well as other books on number theory and mathematical 
analysis. 
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Cg (x) 
FO) 
The part of the graph of f(x) that satisfies 


F(x) < Cg(x) is shown in color. 
gx) 


fŒ) < Ce(x) for x > k 


FIGURE 2 The function f(x) is O(g(x)). 


Example 2 illustrates how big-O notation is used to estimate the growth of functions. 


EXAMPLE 2 Show that 7x? is O(x?). 


Solution: Note that when x > 7, we have 7x” < x°. (We can obtain this inequality by multiplying 
both sides of x > 7 by x.) Consequently, we can take C = 1 and k = 7 as witnesses to establish 
the relationship 7x2 is Ox). Alternatively, when x > 1, we have 7x? < 7x3, so that C = 7 and 
k = 1 are also witnesses to the relationship 7x is O(x*). < 


DONALD E. KNUTH (BORN 1938) Knuth grew up in Milwaukee, where his father taught bookkeeping at 
a Lutheran high school and owned a small printing business. He was an excellent student, earning academic 
achievement awards. He applied his intelligence in unconventional ways, winning a contest when he was in the 
eighth grade by finding over 4500 words that could be formed from the letters in “Ziegler’s Giant Bar.” This 
won a television set for his school and a candy bar for everyone in his class. 
Knuth had a difficult time choosing physics over music as his major at the Case Institute of Technology. 
He then switched from physics to mathematics, and in 1960 he received his bachelor of science degree, si- 
P multaneously receiving a master of science degree by a special award of the faculty who considered his work 
Courtesy of Stanford outstanding. At Case, he managed the basketball team and applied his talents by constructing a formula for 
University News Service the value of each player. This novel approach was covered by Newsweek and by Walter Cronkite on the CBS 
television network. Knuth began graduate work at the California Institute of Technology in 1960 and received 
his Ph.D. there in 1963. During this time he worked as a consultant, writing compilers for different computers. 

Knuth joined the staff of the California Institute of Technology in 1963, where he remained until 1968, when he took a job as a 
full professor at Stanford University. He retired as Professor Emeritus in 1992 to concentrate on writing. He is especially interested 
in updating and completing new volumes of his series The Art of Computer Programming, a work that has had a profound influence 
on the development of computer science, which he began writing as a graduate student in 1962, focusing on compilers. In common 
jargon, “Knuth,” referring to The Art of Computer Programming, has come to mean the reference that answers all questions about 
such topics as data structures and algorithms. 

Knuth is the founder of the modern study of computational complexity. He has made fundamental contributions to the subject of 
compilers. His dissatisfaction with mathematics typography sparked him to invent the now widely used TeX and Metafont systems. 
TeX has become a standard language for computer typography. Two of the many awards Knuth has received are the 1974 Turing 
Award and the 1979 National Medal of Technology, awarded to him by President Carter. 

Knuth has written for a wide range of professional journals in computer science and in mathematics. However, his first publi- 
cation, in 1957, when he was a college freshman, was a parody of the metric system called “The Potrzebie Systems of Weights and 
Measures,” which appeared in MAD Magazine and has been in reprint several times. He is a church organist, as his father was. He 
is also a composer of music for the organ. Knuth believes that writing computer programs can be an aesthetic experience, much like 
writing poetry or composing music. 

Knuth pays $2.56 for the first person to find each error in his books and $0.32 for significant suggestions. If you send him 
a letter with an error (you will need to use regular mail, because he has given up reading e-mail), he will eventually inform you 
whether you were the first person to tell him about this error. Be prepared for a long wait, because he receives an overwhelming 
amount of mail. (The author received a letter years after sending an error report to Knuth, noting that this report arrived several 
months after the first report of this error.) 
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Remark: In Example 2 we did not choose the smallest possible power of x the reference function 
in the big-O estimate. Note that 7x is also big-O of x? and x? grows much slower than x°. In 
fact, x? would be the smallest possible power of x suitable as the reference function in the big-O 
estimate. 


Example 3 illustrates how to show that a big-O relationship does not hold. 


Show that n? is not O(n). 


Solution: To show that n? is not O(n), we must show that no pair of witnesses C and k exist such 
that n2? < Cn whenever n > k. We will use a proof by contradiction to show this. 

Suppose that there are constants C and k for which n? < Cn whenever n > k. Observe that 
when n > 0 we can divide both sides of the inequality n? < Cn by n to obtain the equivalent 
inequality n < C. However, no matter what C and k are, the inequality n < C cannot hold for 
all n with n > k. In particular, once we set a value of k, we see that when n is larger than the 
maximum of k and C, it is not true that n < C even though n > k. This contradiction shows that 
n? is not O(n). < 


Example 2 shows that 7x? is O(x°). Is it also true that x? is O(7x")? 


Solution: To determine whether x? is O(7x7), we need to determine whether witnesses C and 
k exist, so that x° < C(7x?) whenever x > k. We will show that no such witnesses exist using a 
proof by contradiction. 

If C and k are witnesses, the inequality L< Cx) holds for all x > k. Observe that 
the inequality x? < C(7x’) is equivalent to the inequality x < 7C, which follows by divid- 
ing both sides by the positive quantity x?. However, no matter what C is, it is not the case 
that x < 7C for all x > k no matter what k is, because x can be made arbitrarily large. It fol- 
lows that no witnesses C and k exist for this proposed big-O relationship. Hence, x° is not 
O(7x?). < 


3.2.3 Big-O Estimates for Some Important Functions 


Polynomials can often be used to estimate the growth of functions. Instead of analyzing the 
growth of polynomials each time they occur, we would like a result that can always be used 
to estimate the growth of a polynomial. Theorem | does this. It shows that the leading term 
of a polynomial dominates its growth by asserting that a polynomial of degree n or less 
is O(x"). 


Let f(x) = a,x" + a,x"! + ++ +.a,x + do, where ap, ay, ..., 4,1, 4, are real numbers. Then 


f(x) is O}"). 
Proof: Using the triangle inequality (see Exercise 9 in Section 1.8), if x > 1 we have 
IFO] = a,x" + a,x"! + + ax + agl 
< a, |x" + lap lx! + + layla + laol 


= x" (Ia,,| + la, |/x + E leer + |ay|/x") 


<x" (la, + lapal + + lay | + laol) : 
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This shows that 
IF| < Cx", 


where C = |a,| + |a,_,;| +++: + |a9| whenever x > 1. Hence, the witnesses C = |a,| + |a,_1| 
+++» + |ag| and k = 1 show that f(x) is O(x"). 


We now give some examples involving functions that have the set of positive integers as 
their domains. 


How can big-O notation be used to estimate the sum of the first positive integers? 


Solution: Because each of the integers in the sum of the first n positive integers does not exceed 
n, it follows that 


1+2+--¢n<ntnt-tn=n’. 


From this inequality it follows that 1 +2 +3 + =- + n is O(n’), taking C = 1 and k = 1 as wit- 


nesses. (In this example the domains of the functions in the big-O relationship are the set of 
positive integers.) < 


In Example 6 big-O estimates will be developed for the factorial function and its logarithm. 
These estimates will be important in the analysis of the number of steps used in sorting proce- 
dures. 


Give big-O estimates for the factorial function and the logarithm of the factorial function, where 
the factorial function f(n) = n! is defined by 


ni=1-2-3- 06 -n 
whenever n is a positive integer, and 0! = 1. For example, 

W=1, 2!=1-2=2, 31=1-2-3=6 4$=1-2-3-4=24. 
Note that the function n! grows rapidly. For instance, 


20! = 2,432,902,008,176,640,000. 


Solution: A big-O estimate for n! can be obtained by noting that each term in the product does 
not exceed n. Hence, 


ni=z1-2-3. «nn 
<n-n-n: os -n 
=n". 


This inequality shows that n! is O(n”), taking C = 1 and k = 1 as witnesses. Taking logarithms 
of both sides of the inequality established for n!, we obtain 


logn! < logn” = n logn. 


This implies that logn! is O(n logn), again taking C = 1 and k = 1 as witnesses. < 
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In Section 5.1 , we will show that n < 2” whenever n is a positive integer. Show that this in- 
equality implies that n is O(2”), and use this inequality to show that log n is O(n). 


Solution: Using the inequality n < 2”, we quickly can conclude that n is O(2”) by taking k = 
C = 1 as witnesses. Note that because the logarithm function is increasing, taking logarithms 
(base 2) of both sides of this inequality shows that 


logn <n. 
It follows that 
logn is O(n). 


(Again we take C = k = 1 as witnesses.) 
If we have logarithms to a base b, where b is different from 2, we still have log, n is O(n) 
because 


whenever n is a positive integer. We take C = 1/ log b and k = 1 as witnesses. (We have used 
Theorem 3 in Appendix 2 to see that log, n = logn / log b.) < 


As mentioned before, big-O notation is used to estimate the number of operations needed to 
solve a problem using a specified procedure or algorithm. The functions used in these estimates 
often include the following: 


1, logn, n, nlogn, ne, 2", n! 


Using calculus it can be shown that each function in the list is smaller than the succeeding 
function, in the sense that the ratio of a function and the succeeding function tends to zero as n 
grows without bound. Figure 3 displays the graphs of these functions, using a scale for the values 
of the functions that doubles for each successive marking on the graph. That is, the vertical scale 
in this graph is logarithmic. 


4096 
2048 
1024 


FIGURE 3 A display of the growth of functions commonly used in big-O estimates. 
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USEFUL BIG-O ESTIMATES INVOLVING LOGARITHMS, POWERS, AND EXPONEN- 
TIAL FUNCTIONS We now give some useful facts that help us determine whether big-O 
relationships hold between pairs of functions when each of the functions is a power of a loga- 
rithm, a power, or an exponential function of the form b” where b > 1. Their proofs are left as 
Exercises 57—62 for readers skilled with calculus. 

Theorem 1 shows that if f(n) is a polynomial of degree d or less, then f (n) is O(n“). Applying 
this theorem, we see that if d > c > 1, then‘ is O(n“). We leave it to the reader to show that the 
reverse of this relationship does not hold. Putting these facts together, we see that if d > c > 1, 
then 


n° is O(n), but n? is not O(n‘). 


In Example 7 we showed that log, n is O(n) whenever b > 1. More generally, whenever b > 1 
and c and d are positive, we have 


(log, n)° is O(n“), but n? is not (O(log, n)°). 


This tells us that every positive power of the logarithm of n to the base b, where b > 1, is big-O 
of every positive power of n, but the reverse relationship never holds. 

In Example 7, we also showed that n is O(2”). More generally, whenever d is positive and 
b > 1, we have 


n? is O(b"), but b” is not O(n®). 


This tells us that every power of n is big-O of every exponential function of n with a base that 
is greater than one, but the reverse relationship never holds. Furthermore, when c > b > 1 we 
have 


b” is O(c"), but c” is not O(b”). 


This tells us that if we have two exponential functions with different bases greater than one, one 
of these functions is big-O of the other if and only if its base is smaller or equal. 
Finally, we note that if c > 1, we have 


c” is O(n!), but n! is not O(c"). 


We can use the big-O estimates discussed here to help us order the growth of different 
functions, as Example 8 illustrates. 


Arrange the functions f, (n) = 84/n, a(n) = (log n)*, fy(n) = 2n logn, f(n) = n!, fs(n) = (1.1), 
and f(n) = n? in a list so that each function is big-O of the next function. 


Solution: From the big-O estimates described in this subsection, we see that f,(n) = (log n)? 
is the slowest growing of these functions. (This follows because logn grows slower than any 
positive power of n.) The next three functions, in order, are f; (n) = 8y/n = f;(n) = 2nlogn, and 
fg(n) = n?. (We know this because f, (n) = 8n!/2, f(n) = 2n log n is a function that grows faster 
than n but slower than n° for every c > 1, and f4(n) = n? is of the form n° where c = 2.) The next 
function in the list is f5(7) = (1.1)”, because it is an exponential function with base 1.1. Finally, 
fa(n) = n! is the fastest growing function on the list, because f(n) = n! grows faster than any 
exponential function of n. < 
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3.2.4 The Growth of Combinations of Functions 


Many algorithms are made up of two or more separate subprocedures. The number of steps 
used by a computer to solve a problem with input of a specified size using such an algorithm is 
the sum of the number of steps used by these subprocedures. To give a big-O estimate for the 
number of steps needed, it is necessary to find big-O estimates for the number of steps used by 
each subprocedure and then combine these estimates. 

Big-O estimates of combinations of functions can be provided if care is taken when different 
big-O estimates are combined. In particular, it is often necessary to estimate the growth of the 
sum and the product of two functions. What can be said if big-O estimates for each of two 
functions are known? To see what sort of estimates hold for the sum and the product of two 
functions, suppose that f(x) is O(g,(x)) and f,(x) is O(g,(x)). 

From the definition of big-O notation, there are constants C,, C,, k,, and k, such that 


AWI < Cylg,@| 


when x > k,, and 


IA@| < Calel 


when x > k,. To estimate the sum of f (x) and f,(x), note that 


IA, + AOD = 1A@) +0)! 
< |A@|+1f@)| using the triangle inequality |a + b| < la] + |D|. 


When x is greater than both k, and k,, it follows from the inequalities for | f, (x)| and | f,(x)| that 


IAI + LA! Clew + Cle) 
< Cill + Cleo 
= (C1 + Clg) 
= Cle@)|, 
where C = C, + C, and g(x) = max(|g,(x)|, Ig2(x)|). [Here max(a, b) denotes the maximum, or 
larger, of a and b.] 


This inequality shows that |(f, + f,)(x)| < Clg(x)| whenever x > k, where k = max(k,, k3). 
We state this useful result as Theorem 2. 


Suppose that f, (x) is O(g,(x)) and that f,(x) is O(g,(x)). Then (fi +5)(x) is O(g(x)), where 
g(x) = (max(|g;(%)|, |820|) for all x. 


We often have big-O estimates for f, and fọ in terms of the same function g. In this situation, 
Theorem 2 can be used to show that (f; +/5)(x) is also O(g(x)), because max(g(x), g(x)) = g(x). 
This result is stated in Corollary 1. 


Suppose that f, (x) and fa (x) are both O(g(x)). Then (fi + f)(x) is O(g(x)). 
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Q and © are the Greek 
uppercase letters omega 
and theta, respectively. 


In a similar way big-O estimates can be derived for the product of the functions f, and fz. 
When x is greater than max(k,, k,) it follows that 


AAI! = IA CONA)| 
< Gla @|Crlg2Q)| 
< CCa) 
< Cl(8182)@)I, 


where C = C,C,. From this inequality, it follows that f, @f>(x) is O(g8182(x)), because there are 
constants C and k, namely, C = C, C, and k = max(k,, k3), such that |(f,)@)| < Cl Lgl 
whenever x > k. This result is stated in Theorem 3. 


Suppose that f, (x) is O(g,(x)) and fa (x) is Olg (x)). Then (f,f,)(x) is O(g,(«)go(x)). 


The goal in using big-O notation to estimate functions is to choose a function g(x) as simple 
as possible, that grows relatively slowly so that f(x) is O(g(x)). Examples 9 and 10 illustrate how 
to use Theorems 2 and 3 to do this. The type of analysis given in these examples is often used 
in the analysis of the time used to solve problems using computer programs. 


Give a big-O estimate for f(n) = 3nlog(n!) + (n? + 3) logn, where n is a positive integer. 


Solution: First, the product 3n log(n!) will be estimated. From Example 6 we know that log(n!) 
is O(n logn). Using this estimate and the fact that 3n is O(n), Theorem 3 gives the estimate that 
3n log(n!) is O(n? logn). 

Next, the product (n? + 3) log n will be estimated. Because (n? + 3) < 2n? when n > 2, it 
follows that n? + 3 is O(n”). Thus, from Theorem 3 it follows that (n? + 3) logn is O(n? logn). 
Using Theorem 2 to combine the two big-O estimates for the products shows that f(n) = 
3nlog(n!) + (n? + 3) logn is O(n? logn). 


Give a big-O estimate for f(x) = (x + 1) log(x? + 1) + 3x’. 


Solution: First, a big-O estimate for (x + 1) log(x? + 1) will be found. Note that (x + 1) is O(x). 
Furthermore, x? + 1 < 2x? when x > 1. Hence, 


log(x? + 1) < log(2x’) = log 2 + log x = log2 + 2logx < 3logx, 


if x > 2. This shows that log(x? + 1) is O(log x). 

From Theorem 3 it follows that (x + 1) log(x* + 1) is O(x log x). Because 3x is O(x), The- 
orem 2 tells us that f(x) is O(max(x log x, x*)). Because xlogx < x’, for x > 1, it follows that 
f(x) is OX). < 


3.2.5 Big-Omega and Big-Theta Notation 


Big-O notation is used extensively to describe the growth of functions, but it has limitations. 
In particular, when f(x) is O(g(x)), we have an upper bound, in terms of g(x), for the size of 
f(x) for large values of x. However, big-O notation does not provide a lower bound for the size 
of f(x) for large x. For this, we use big-Omega (big-Q) notation. When we want to give both 
an upper and a lower bound on the size of a function f(x), relative to a reference function g(x), 
we use big-Theta (big-©) notation. Both big-Omega and big-Theta notation were introduced 
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by Donald Knuth in the 1970s. His motivation for introducing these notations was the common 
misuse of big-O notation when both an upper and a lower bound on the size of a function are 
needed. We now define big-Omega notation and illustrate its use. After doing so, we will do 
the same for big-Theta notation. 


Let f and g be functions from the set of integers or the set of real numbers to the set of real 
numbers. We say that f(x) is Q(g(x)) if there are constants C and k with C positive such that 


Fœ 2 Clg@)| 


whenever x > k. [This is read as “f(x) is big-Omega of g(x).”] 


There is a strong connection between big-O and big-Omega notation. In particular, f(x) is 
Q(g(x)) if and only if g(x) is O(f(x)). We leave the verification of this fact as a straightforward 
exercise for the reader. 


The function f(x) = 8x + 5x? + 7 is Q(g(x)), where g(x) is the function g(x) = x°. This is easy 
to see because f(x) = 8x? + 5x? +7 > 8x for all positive real numbers x. This is equivalent 
to saying that g(x) = x is O(8x* + 5x? + 7), which can be established directly by turning the 
inequality around. 


Often, it is important to know the order of growth of a function in terms of some relatively 
simple reference function such as x” when n is a positive integer or c*, where c > 1. Knowing 
the order of growth requires that we have both an upper bound and a lower bound for the size 
of the function. That is, given a function f(x), we want a reference function g(x) such that f(x) 
is O(g(x)) and f(x) is Q(g(x)). Big-Theta notation, defined as follows, is used to express both of 
these relationships, providing both an upper and a lower bound on the size of a function. 


Let f and g be functions from the set of integers or the set of real numbers to the set of 
real numbers. We say that f(x) is O(g(x)) if f(x) is O(g(x)) and f(x) is Q(g(x)). When f(x) is 
O(g(x)), we say that f is big-Theta of g(x), that f(x) is of order g(x), and that f(x) and g(x) 
are of the same order. 


When f(x) is O(g(x)), it is also the case that g(x) is O(f(x)). Also note that f(x) is O(g(x)) if 
and only if f(x) is O(g(x)) and g(x) is OC f(x)) (see Exercise 31). Furthermore, note that f(x) is 
O©(g(x)) if and only if there are positive real numbers C} and C, and a positive real number k 
such that 


Cill < Ifl Cols! 


whenever x > k. The existence of the constants C,, C,, and k tells us that f(x) is Q(g(x)) and 
that f(x) is O(g(x)), respectively. 

Usually, when big-Theta notation is used, the function g(x) in O(g(x)) is a relatively simple 
reference function, such as x", c*, log x, and so on, while f(x) can be relatively complicated. 


We showed (in Example 5) that the sum of the first n positive integers is O(n*). Determine 
whether this sum is of order n? without using the summation formula for this sum. 


Solution: Let f(n) = 1 +2 +3 +++ +n. Because we already know that f(n) is O(n’), to show 
that f(n) is of order n? we need to find a positive constant C such that f(n) > Cn? for sufficiently 
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large integers n. To obtain a lower bound for this sum, we can ignore the first half of the terms. 
Summing only the terms greater than [n/2], we find that 
1+2+- +n > [n/2]+([n/2|+1)+ +n 
> [n/2] + [n/2] ++ + [n/2] 
= (n — [n/2] + 1) [n/2] 
2 (n/2)(n/2) 
=n /4. 


This shows that f(n) is Q(n?). We conclude that f(n) is of order n?, or in symbols, f(n) 
is O(n’). < 


Remark: Note that we can also show that f(n) = Èi is O(nĉ) using the closed formula 
yy, = n(n + 1)/2 from Table 2 in Section 2.4 and derived in Exercise 37(b) of that section. 


i=l 


Show that 3x? + 8xlog x is O(x’). 


Solution: Because 0 < 8xlogx < 8x*, it follows that 3x7+8xlogx < 11x? for x> 1. 
Consequently, 3x? + 8xlogx is O(x*). Clearly, x? is O(3x* + 8xlogx). Consequently, 
3x* + 8x log x is Q(x’). < 


One useful fact is that the leading term of a polynomial determines its order. For example, 
if f(x) = 3x° + xt + 17x2 + 2, then f(x) is of order x°. This is stated in Theorem 4, whose proof 
is left as Exercise 50. 


THEOREM 4 Let f@ =a,x"+a,_,x"! ++ +4a,x+4 9, where do, a),...,a, are real numbers with 
a, # 0. Then f(x) is of order x”. 
EXAMPLE 14 The polynomials 3x8 + 10x7 + 221x? + 1444, x!° — 18x4 — 10,112, and —x* + 40,001x°8 + 
100,003x are of orders x8, x!°, and x” , respectively. < 
Unfortunately, as Knuth observed, big-O notation is often used by careless writers and 
speakers as if it had the same meaning as big-Theta notation. Keep this in mind when you see 
big-O notation used. The recent trend has been to use big-Theta notation whenever both upper 
and lower bounds on the size of a function are needed. 
Exercises 
In Exercises 1-14, to establish a big-O relationship, find wit- 2. Determine whether each of these functions is O(x2). 
nesses C and k such that |f(x)| < C|g(x)| whenever x > k. a) f(x) =17x + 11 b) f(x) = x? + 1000 
= el 
1. Determine whether each of these functions is O(x). i Ay = os r T B K fx] 
a) fx)=10 b) f@) =3x+7 
ec) fa) =x +x+l d) f(x) = 5logx 3. Use the definition of “f(x) is O(g(x))” to show that x* + 
e) f(x) = [x] f) fœ) = [x/2] 9x3 + 4x +7 is O(x*). 


nN 


15. 
16. 
17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


. Use the definition of “f(x) is O(g(x))” to show that 


2* + 17 is O(3*). 


. Show that (x? + 1)/(x + 1) is Ox). 
. Show that (x3 + 2x)/(2x + 1) is O(2?). 
. Find the least integer n such that f(x) is O(x”) for each of 


these functions. 

a) f(x) = 2x3 +x? logx 

b) fŒ = 3x? + (log x)4 

0 fM=044 24 )/e4+) 
d) fŒ = G4 + 5logx)/(x4 +1) 


. Find the least integer n such that f(x) is O(x”) for each of 


these functions. 

a) f(x) = 2x? + x logx 

b) f(@®) = 3x + (log x)* 

o0 f®=04424)/O44+) 
d) fŒ = + 5logx)/(x4 +1) 


. Show that x? + 4x + 17 is O(°) but that x? is not Ox? + 


4x +17). 


. Show that x? is O(x*) but that x* is not O(x3). 

. Show that 3x4 + 1 is O(x4 /2) and x4/2 is O(3x* + 1). 

. Show that x log x is O(x*) but that x? is not O(x log x). 

. Show that 2” is O(3") but that 3” is not O(2”). (Note that 


this is a special case of Exercise 60.) 


. Determine whether x? is O(g(x)) for each of these func- 
tions g(x). 
a) g0) =x b) g0) =x 
o ga) =r d) ga =x 4x4 
e) ga) =3* f) ga) = /2 
Explain what it means for a function to be O(1). 


Show that if f(x) is O(x), then f(x) is O(x’). 

Suppose that f(x), g(x), and h(x) are functions such that 
F(x) is O(g(x)) and g(x) is O(h(x)). Show that f(x) is 
O(h(a)). 

Let k be a positive integer. Show that 1‘ +2% + -+ n* 
is O(n'*!), 

Determine whether each of the functions 2”*! and 27” 
is O(2"). 

Determine whether each of the functions log(n + 1) and 
log(n? + 1) is Oogn). 

Arrange the functions yn, 1000 logn, nlogn, 2n!, 2”, 3”, 
and n? /1,000,000 in a list so that each function is big-O 
of the next function. 

Arrange the functions (1.5)", 100, (log n}, /nlog n, 
10", (n!)*, and n°? + n°? in a list so that each function 
is big-O of the next function. 

Suppose that you have two different algorithms for solv- 
ing a problem. To solve a problem of size n, the first 
algorithm uses exactly n(logn) operations and the sec- 
ond algorithm uses exactly n?/? operations. As n grows, 
which algorithm uses fewer operations? 

Suppose that you have two different algorithms for solv- 
ing a problem. To solve a problem of size n, the first 
algorithm uses exactly n*2” operations and the second 
algorithm uses exactly n! operations. As n grows, which 
algorithm uses fewer operations? 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 
37. 
38. 


39. 


40. 
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Give as good a big-O estimate as possible for each of 

these functions. 

a) (n? +8)(n+ 1) b) (nlogn + n?n? + 2) 

c) (n! +20 + log(n? + 1)) 

Give a big-O estimate for each of these functions. For the 

function g in your estimate f(x) is O(g(x)), use a simple 

function g of smallest order. 

a) (n° +n? log n)(logn+1) + (17 logn+19)(n3 +2) 

b) (2" + n?n? + 3") 

c) (n” +72" +5")(n! + 5") 

Give a big-O estimate for each of these functions. For 

the function g in your estimate that f(x) is O(g(x)), use a 

simple function g of the smallest order. 

a) nlog(n? + 1) +77 logn 

b) (nlogn+ 1)? + dogn + I(r? + 1) 

c) n” +n” 

For each function in Exercise 1, determine whether that 

function is Q(x) and whether it is O(x). 

For each function in Exercise 2, determine whether that 

function is Q(x?) and whether it is O(2x7). 

Show that each of these pairs of functions are of the same 

order. 

a) 3x+7,x 

b) 2x7 +x-7,x° 

c) [x+1/2|,x 

d) log(x’ + 1), log, x 

e) log) x, log, x 

Show that f(x) is O(g(x)) if and only if f(x) is O(g(x)) and 

(x) is OF). 

Show that if f(x) and g(x) are functions from the set 

of real numbers to the set of real numbers, then f(x) is 

O(g(x)) if and only if g(x) is Q(f@)). 

Show that if f(x) and g(x) are functions from the set 

of real numbers to the set of real numbers, then f(x) is 

O©(g(x)) if and only if there are positive constants k, Cy, 

and C, such that Cile < F| < C.|lg@)| whenever 

x>k. 

a) Show that 3x? +x+ 1 is @(3x?) by directly finding 
the constants k, C}, and C, in Exercise 33. 

b) Express the relationship in part (a) using a picture 
showing the functions 3x? + x + 1, C 1 3x2, and C: 
3x, and the constant k on the x-axis, where C p Cr, 
and k are the constants you found in part (a) to show 
that 3x? +x + 1 is O(3x’). 

Express the relationship f(x) is O(g(x)) using a picture. 

Show the graphs of the functions f(x), C,|g(x)|, and 

C,|g(x)|, as well as the constant k on the x-axis. 

Explain what it means for a function to be Q(1). 

Explain what it means for a function to be @(1). 

Give a big-O estimate of the product of the first n odd 

positive integers. 

Show that if f and g are real-valued functions such that 

F(x) is O(g(x)), then for every positive integer n, f”(x) is 

O(g"(x)). [Note that fœ) = f(@)".] 

Show that for all real numbers a and b with a > 1 and 

b> 1, if f(x) is O(log, x), then f(x) is O(log, x). 
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41. Suppose that f(x) is O(g(x)), where f and g are increas- 
ing and unbounded functions. Show that log |f(x)| is 


O(log |g(x))). 


42. Suppose that f(x) is O(g(x)). Does it follow that 2/ is 
O28)? 


43. Let fi Œœ) and f(x) be functions from the set of real num- 
bers to the set of positive real numbers. Show that if f, (x) 
and f,(x) are both @(g(x)), where g(x) is a function from 
the set of real numbers to the set of positive real numbers, 
then fi œ) + fo(x) is O(g(x)). Is this still true if fı (x) and 
fa Œœ) can take negative values? 


44. Suppose that f(x), g(x), and A(x) are functions such that 
f(x) is O(g(x)) and g(x) is O(A(x)). Show that f(x) is 
O(h(~)). 

45. If fi (x) and f(x) are functions from the set of positive in- 
tegers to the set of positive real numbers and f,(x) and 
fa Œœ) are both O(g(x)), is (fi — f)(x) also O(g(x))? Either 
prove that it is or give a counterexample. 


46. Show that if fi(x) and f(x) are functions from the 
set of positive integers to the set of real numbers and 
f,@) is O(g;@)) and h) is O(g,(~)), then (F) is 
O(8182)()). 

47. Find functions f and g from the set of positive integers to 
the set of real numbers such that f(n) is not O(g(n)) and 
g(n) is not O(f(n)). 

48. Express the relationship f(x) is Q(g(x)) using a picture. 


Show the graphs of the functions f(x) and Cg(x), as well 
as the constant k on the real axis. 


49. Show that iff, (x) is O(g, (x)). hx) is O(g.(x)), and h œ) + 
0 and g,(x) # 0 for all real numbers x > 0, then (f, /f,)() 
is O((g /82)()). 


50. Show that if fœ) = a,x" + a,x"! + + ax + do, 
where dy, a), --., aA,- and a, are real numbers and a, # 
0, then f(x) is Ox”). 

Big-O, big-Theta, and big-Omega notation can be extended to 

functions in more than one variable. For example, the state- 

ment f(x, y) is O(g(x, y)) means that there exist constants C, 

k,, and k, such that [f(x y)| < C|g(x, y)| whenever x > k; and 

y > ky. 

51. Define the statement f(x, y) is O(g(, y)). 

52. Define the statement f(x, y) is Q(g(, y)). 

53. Show that (x? + xy + xlog y} is O(x*y3). 

54. Show that xy? + x4y4 + Py? is Qy’). 

55. Show that |xy] is O(xy). 

56. Show that [xy] is Q(xy). 


57. (Requires calculus) Show that if c > d > 0, then nf is 
O(n‘), but n° is not O(n). 

58. (Requires calculus) Show that if b > 1 and c and d are 
positive, then (log, 1)° is O(n?), but n is not O((log, 7)°). 


59. (Requires calculus) Show that if d is positive and b > 1, 
then n? is O(b"), but b” is not O(n”). 


60. (Requires calculus) Show that if c > b > 1, then b” is 
O(c"), but c” is not O(b"). 

61. (Requires calculus) Show that if c > 1, then c” is O(n!), 
but n! is not O(c"). 

62. (Requires calculus) Prove or disprove that (2n)! is O(n!). 


The following problems deal with another type of asymptotic 
notation, called little-o notation. Because little-o notation is 
based on the concept of limits, a knowledge of calculus is 
needed for these problems. We say that f(x) is o(g(x)) [read 
f(x) is “little-oh” of g(x)], when 

lim fœ =0 
x>% g(x) 


63. (Requires calculus) Show that 
a) x? is o(x°). 

c) x? is 0(2*). 

64. (Requires calculus) 

a) Show that if f(x) and g(x) are functions such that f(x) 
is o(g(x)) and c is a constant, then cf(x) is o(g(x)), 
where (cf)(x) = cf). 

b) Show that if f;(x), (x), and g(x) are functions such 
that f(x) is o(g(x)) and f,(x) is o(g(x)), then (fi + 
ID) is o(g(x)), where (f, + =f) +h. 

65. (Requires calculus) Represent pictorially that xlogx is 
o(x”) by graphing x log x, x”, and x log x/x?. Explain how 
this picture shows that x log x is o(x7). 

66. (Requires calculus) Express the relationship f(x) is 
o(g(x)) using a picture. Show the graphs of f(x), g(x), and 
£09/g. 

*67. (Requires calculus) Suppose that f(x) is o(g(x)). Does it 

follow that 2/ is 0(28)? 

* 68. (Requires calculus) Suppose that f(x) is o(g(x)). Does it 
follow that log |[f(x)| is odog |g(x)|)? 

69. (Requires calculus) The two parts of this exercise de- 
scribe the relationship between little-o and big-O nota- 
tion. 

a) Show that if f(x) and g(x) are functions such that f(x) 
is o(g(x)), then f(x) is O(g(x)). 

b) Show that if f(x) and g(x) are functions such that f(x) 
is O(g(x)), then it does not necessarily follow that f(x) 
is o(g(x)). 

70. (Requires calculus) Show that if f(x) is a polynomial of 
degree n and g(x) is a polynomial of degree m where 
m > n, then f(x) is o(g(x)). 

71. (Requires calculus) Show that if f, (x) is O(g(x)) and fa (x) 
is o(g(x)), then f,(x) +(x) is O). 

72. (Requires calculus) Let H,, be the nth harmonic number 


b) xlogx is ox). 
d) x? +x + 1 is not o(x°). 


1 1 1 
H,=14+s42404-. 
a 22°33 n 


Show that H, is O(logn). [Hint: First establish the in- 
equality 


* 73, 
ES 74. 


*75. 


Let 


by showing that the sum of the areas of the rectangles of 
height 1/j with base from j — 1 to j, for j = 2, 3, ..., n, is 
less than the area under the curve y = 1 /x from 2 to n.] 


Show that n logn is O(log n!). 


Determine whether logn! is O(n log). Justify your an- 
Swer. 


Show that log n! is greater than (nlogn)/4 for 
n>4. [Hint: Begin with the inequality n! > 
n(n — 1)(n — 2) ++ [n/2].] 


f(x) and g(x) be functions from the set of real 


numbers to the set of real numbers. We say that the 
functions f and g are asymptotic and write f(x) ~ g(x) 
if lim „ns f@)/g() = 1. 
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(Requires calculus) For each of these pairs of functions, 
determine whether f and g are asymptotic. 
a) f(@®) =x +3x4+7, gx) =x + 10 
b) fŒ =x logx, g(x) = xX 
c) fœ) =x + log3x* + 7), 
g(x) = (x7 + 17x +3) 
d) f@ = 42x? 4x4 D, 
a(x) = O44 4272 4x4 19. 
(Requires calculus) For each of these pairs of functions, 
determine whether f and g are asymptotic. 
a) fœ) = log? + 1), g = logx 
b) f@) = 2, g@) = 27 
©) f(x) = 2%, gx) =2" 
d) f(x) = Qetatl g(x) = px 42x 


3.3.1 Introduction 


When does an algorithm provide a satisfactory solution to a problem? First, it must always pro- 
duce the correct answer. How this can be demonstrated will be discussed in Chapter 5. Second, 
it should be efficient. The efficiency of algorithms will be discussed in this section. 

How can the efficiency of an algorithm be analyzed? One measure of efficiency is the time 
used by a computer to solve a problem using the algorithm, when input values are of a specified 
size. A second measure is the amount of computer memory required to implement the algorithm 
when input values are of a specified size. 

Questions such as these involve the computational complexity of the algorithm. An anal- 
ysis of the time required to solve a problem of a particular size involves the time complexity 
of the algorithm. An analysis of the computer memory required involves the space complexity 
of the algorithm. Considerations of the time and space complexity of an algorithm are essential 
when algorithms are implemented. It is important to know whether an algorithm will produce 
an answer in a microsecond, a minute, or a billion years. Likewise, the required memory must 
be available to solve a problem, so that space complexity must be taken into account. 

Considerations of space complexity are tied in with the particular data structures used to 
implement the algorithm. Because data structures are not dealt with in detail in this book, space 
complexity will not be considered. We will restrict our attention to time complexity. 


3.3.2 Time Complexity 


The time complexity of an algorithm can be expressed in terms of the number of operations 
used by the algorithm when the input has a particular size. The operations used to measure time 
complexity can be the comparison of integers, the addition of integers, the multiplication of 
integers, the division of integers, or any other basic operation. 

Time complexity is described in terms of the number of operations required instead of ac- 
tual computer time because of the difference in time needed for different computers to perform 
basic operations. Moreover, it is quite complicated to break all operations down to the basic bit 
operations that a computer uses. Furthermore, the fastest computers in existence can perform 
basic bit operations (for instance, adding, multiplying, comparing, or exchanging two bits) in 
107}! second (10 picoseconds), but personal computers may require 1078 second (10 nanosec- 
onds), which is 1000 times as long, to do the same operations. 
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EXAMPLE 1 


Extra » 
Examples 


EXAMPLE 2 


EXAMPLE 3 


We illustrate how to analyze the time complexity of an algorithm by considering Algo- 
rithm 1 of Section 3.1, which finds the maximum of a finite set of integers. 


Describe the time complexity of Algorithm 1 of Section 3.1 for finding the maximum element 
in a finite set of integers. 


Solution: The number of comparisons will be used as the measure of the time complexity of the 
algorithm, because comparisons are the basic operations used. 

To find the maximum element of a set with n elements, listed in an arbitrary order, the 
temporary maximum is first set equal to the initial term in the list. Then, after a comparison 
i < n has been done to determine that the end of the list has not yet been reached, the temporary 
maximum and second term are compared, updating the temporary maximum to the value of 
the second term if it is larger. This procedure is continued, using two additional comparisons 
for each term of the list—one i < n, to determine that the end of the list has not been reached 
and another max < a;, to determine whether to update the temporary maximum. Because two 
comparisons are used for each of the second through the nth elements and one more comparison 
is used to exit the loop when i =n + 1, exactly 2(n — 1) + 1 = 2n — 1 comparisons are used 
whenever this algorithm is applied. Hence, the algorithm for finding the maximum of a set 
of n elements has time complexity O(n), measured in terms of the number of comparisons used. 
Note that for this algorithm the number of comparisons is independent of particular input of n 
numbers. < 


Next, we will analyze the time complexity of searching algorithms. 


Describe the time complexity of the linear search algorithm (specified as Algortihm 2 in 
Section 3.1). 


Solution: The number of comparisons used by Algorithm 2 in Section 3.1 will be taken as the 
measure of the time complexity. At each step of the loop in the algorithm, two comparisons 
are performed—one i < n, to see whether the end of the list has been reached and one x < a,, 
to compare the element x with a term of the list. Finally, one more comparison i < n is made 
outside the loop. Consequently, if x = a;, 2i + 1 comparisons are used. The most comparisons, 
2n + 2, are required when the element is not in the list. In this case, 2n comparisons are used 
to determine that x is not a;, for i = 1, 2, ..., n, an additional comparison is used to exit the 
loop, and one comparison is made outside the loop. So when x is not in the list, a total of 
2n + 2 comparisons are used. Hence, a linear search requires O(n) comparisons in the worst 
case, because 2n + 2 is O(n). < 


WORST-CASE COMPLEXITY The type of complexity analysis done in Example 2 is a 
worst-case analysis. By the worst-case performance of an algorithm, we mean the largest num- 
ber of operations needed to solve the given problem using this algorithm on input of specified 
size. Worst-case analysis tells us how many operations an algorithm requires to guarantee that 
it will produce a solution. 


Describe the time complexity of the binary search algorithm (specified as Algorithm 3 in 
Section 3.1) in terms of the number of comparisons used (and ignoring the time required to 
compute m = |(i + j)/2] in each iteration of the loop in the algorithm). 


Solution: For simplicity, assume there are n = 2* elements in the list a,, a, ..., &„, Where k is a 
nonnegative integer. Note that k = logn. (If n, the number of elements in the list, is not a power 
of 2, the list can be considered part of a larger list with 2%+! elements, where 2% < n < 2%+!, 
Here 2*t! is the smallest power of 2 larger than n.) 


EXAMPLE 4 
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At each stage of the algorithm, i and j, the locations of the first term and the last term of 
the restricted list at that stage, are compared to see whether the restricted list has more than one 
term. If i < j, a comparison is done to determine whether x is greater than the middle term of 
the restricted list. 

At the first stage the search is restricted to a list with 2‘! terms. So far, two comparisons 
have been used. This procedure is continued, using two comparisons at each stage to restrict 
the search to a list with half as many terms. In other words, two comparisons are used at the 
first stage of the algorithm when the list has 2 elements, two more when the search has been 
reduced to a list with 2%! elements, two more when the search has been reduced to a list with 
2‘-2 elements, and so on, until two comparisons are used when the search has been reduced to a 
list with 2! = 2 elements. Finally, when one term is left in the list, one comparison tells us that 
there are no additional terms left, and one more comparison is used to determine if this term is x. 

Hence, at most 2k + 2 = 2log n + 2 comparisons are required to perform a binary search 
when the list being searched has 2‘ elements. (If n is not a power of 2, the original list is 
expanded to a list with 2‘t! terms, where k = [log n], and the search requires at most 2 [logn] + 
2 comparisons.) It follows that in the worst case, binary search requires O(log n) comparisons. 

Note that in the worst case, 2 log n + 2 comparisons are used by the binary search. Hence, 
the binary search uses O(log n) comparisons in the worst case, because 2 logn + 2 = O(log n). 
From this analysis it follows that in the worst case, the binary search algorithm is more efficient 
than the linear search algorithm, because we know by Example 2 that the linear search algorithm 
has O(n) worst-case time complexity. < 


AVERAGE-CASE COMPLEXITY Another important type of complexity analysis, besides 
worst-case analysis, is called average-case analysis. The average number of operations used 
to solve the problem over all possible inputs of a given size is found in this type of analy- 
sis. Average-case time complexity analysis is usually much more complicated than worst-case 
analysis. However, the average-case analysis for the linear search algorithm can be done without 
difficulty, as shown in Example 4. 


Describe the average-case performance of the linear search algorithm in terms of the average 
number of comparisons used, assuming that the integer x is in the list and it is equally likely 
that x is in any position. 


Solution: By hypothesis, the integer x is one of the integers a,, a>, ..., a, in the list. If x is the 
first term a, of the list, three comparisons are needed, one i < n to determine whether the end 
of the list has been reached, one x + a; to compare x and the first term, and one i < n outside 
the loop. If x is the second term a, of the list, two more comparisons are needed, so that a total 
of five comparisons are used. In general, if x is the ith term of the list a;, two comparisons will 
be used at each of the i steps of the loop, and one outside the loop, so that a total of 2i+ 1 
comparisons are needed. Hence, the average number of comparisons used equals 


So tT beet Qn ly — 20a ean 
n ~ n ` 
Using the formula from line 2 of Table 2 in Section 2.4 (and see Exercise 37(b) of Section 2.4), 


n(n + 1) 


1424+3+4--+n= = 


Hence, the average number of comparisons used by the linear search algorithm (when x is known 
to be in the list) is 


Ann + 1/2 
n 


which is O(n). < 


=n+2, 
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EXAMPLE 5 


EXAMPLE 6 


Remark: In the analysis in Example 4 we assumed that x is in the list being searched. It is also 
possible to do an average-case analysis of this algorithm when x may not be in the list (see 
Exercise 23). 


Remark: Although we have counted the comparisons needed to determine whether we have 
reached the end of a loop, these comparisons are often not counted. From this point on we will 
ignore such comparisons. 


WORST-CASE COMPLEXITY OF TWO SORTING ALGORITHMS We analyze the 
worst-case complexity of the bubble sort and the insertion sort in Examples 5 and 6. 


What is the worst-case complexity of the bubble sort in terms of the number of comparisons 
made? 


Solution: The bubble sort described before Example 4 in Section 3.1 sorts a list by performing 
a sequence of passes through the list. During each pass the bubble sort successively compares 
adjacent elements, interchanging them if necessary. When the ith pass begins, the i — 1 largest 
elements are guaranteed to be in the correct positions. During this pass, n — i comparisons are 
used. Consequently, the total number of comparisons used by the bubble sort to order a list of 
n elements is 


(n=-1)+(n-2)+:+2+1= a 

using a summation formula from line 2 in Table 2 in Section 2.4 (and Exercise 37(b) in 
Section 2.4). Note that the bubble sort always uses this many comparisons, because it con- 
tinues even if the list becomes completely sorted at some intermediate step. Consequently, the 
bubble sort uses (n — 1)n/2 comparisons, so it has @(n”) worst-case complexity in terms of the 
number of comparisons used. < 


What is the worst-case complexity of the insertion sort in terms of the number of comparisons 
made? 


Solution: The insertion sort (described in Section 3.1) inserts the jth element into the correct 
position among the first j — 1 elements that have already been put into the correct order. It does 
this by using a linear search technique, successively comparing the jth element with successive 
terms until a term that is greater than or equal to itis found or it compares a; with itself and stops 
because a; is not less than itself. Consequently, in the worst case, j comparisons are required to 
insert the jth element into the correct position. Therefore, the total number of comparisons used 
by the insertion sort to sort a list of n elements is 


n(n + 1) B 


2434--4+n= 
bon 7 


l, 


using the summation formula for the sum of consecutive integers in line 2 of Table 2 of 
Section 2.4 (and see Exercise 37(b) of Section 2.4), and noting that the first term, 1, is missing 
in this sum. Note that the insertion sort may use considerably fewer comparisons if the smaller 
elements started out at the end of the list. We conclude that the insertion sort has worst-case 
complexity O(n’). < 


Links » 
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In Examples 5 and 6 we showed that both the bubble sort and the insertion sort have 
worst-case time complexity @(n”). However, the most efficient sorting algorithms can sort n 
items in O(nlogn) time, as we will show in Sections 8.3 and 11.1 using techniques we de- 
velop in those sections. From this point on, we will assume that sorting n items can be done in 
O(n log n) time. 

You can run animations found on many different websites that simultaneously run different 
sorting algorithms on the same lists. Doing so will help you gain insights into the efficiency 
of different sorting algorithms. Among the sorting algorithms that you can find are the bubble 
sort, the insertion sort, the shell sort, the merge sort, and the quick sort. Some of these anima- 
tions allow you to test the relative performance of these sorting algorithms on lists of randomly 
selected items, lists that are nearly sorted, and lists that are in reversed order. 


3.3.3 Complexity of Matrix Multiplication 


The definition of the product of two matrices can be expressed as an algorithm for computing 
the product of two matrices. Suppose that C = [c;;] is the m x n matrix that is the product of the 
m X k matrix A = [aj] and the k x n matrix B = [bj]. The algorithm based on the definition of 
the matrix product is expressed in pseudocode in Algorithm 1. 


ALGORITHM 1 Matrix Multiplication. 


procedure matrix multiplication(A, B: matrices) 
fori := 1 to m 
forj := 1 ton 
cj:=0 
for q := 1 to k 
Cy `= Cy + digbaj 
return C {C = [ej] is the product of A and B} 


We can determine the complexity of this algorithm in terms of the number of additions and 
multiplications used. 


How many additions of integers and multiplications of integers are used by Algorithm 1 to 
multiply two n X n matrices with integer entries? 


Solution: There are n? entries in the product of A and B. To find each entry requires a total of n 
multiplications and n — 1 additions. Hence, a total of n> multiplications and n?(n — 1) additions 
are used. < 


Surprisingly, there are more efficient algorithms for matrix multiplication than that given in 
Algorithm 1. As Example 7 shows, multiplying two n x n matrices directly from the definition 
requires O(n?) multiplications and additions. Using other algorithms, two n x n matrices can 
be multiplied using on?) multiplications and additions. (Details of such algorithms can be 
found in [CoLeRiSt09].) 

We can also analyze the complexity of the algorithm we described in Chapter 2 for com- 
puting the Boolean product of two matrices, which we display as Algorithm 2. 
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ALGORITHM 2 The Boolean Product of Zero—One Matrices. 


procedure Boolean product of Zero—One Matrices (A, B: zero—one matrices) 
for i:= 1 tom 
for j:=1ton 
cy = 0 
for q := 1 tok 
Cy = Cy V (aig A bg) 
return C {C = [c;] is the Boolean product of A and B} 


The number of bit operations used to find the Boolean product of two n X n matrices can 
be easily determined. 


How many bit operations are used to find A © B, where A and B are n X n zero—one matrices? 


Solution: There are n? entries in A©B. Using Algorithm 2, a total of n ORs and n ANDs are 
used to find an entry of A © B. Hence, 2n bit operations are used to find each entry. Therefore, 
2n? bit operations are required to compute A ©B using Algorithm 2. < 


MATRIX-CHAIN MULTIPLICATION There is another important problem involving the 
complexity of the multiplication of matrices. How should the matrix-chain A,A, --- A, be 
computed using the fewest multiplications of integers, where A}, Ay, ..., A,, are m, X my, m, X 
m3,...,M, XM,,,; Matrices, respectively, and each has integers as entries? (Because matrix 
multiplication is associative, as shown in Exercise 13 in Section 2.6, the order of the mul- 
tiplication used does not change the product.) Note that m,m,m, multiplications of integers 
are performed to multiply an m; X m, matrix and an m, Xm, matrix using Algorithm 1. 
Example 9 illustrates this problem. 


In which order should the matrices A,, A,, and A;—where A, is 30 x 20, A, is 20 x 40, and 
A, is 40 x 10, all with integer entries—be multiplied to use the least number of multiplications 
of integers? 


Solution: There are two possible ways to compute A, A,A,. These are A,(A,A,) and (A, A,)A3. 

If A, and A; are first multiplied, a total of 20 - 40 - 10 = 8000 multiplications of inte- 
gers are used to obtain the 20 x 10 matrix A,A;. Then, to multiply A; and A,A; requires 
30 - 20 - 10 = 6000 multiplications. Hence, a total of 


8000 + 6000 = 14,000 
multiplications are used. On the other hand, if A, and A, are first multiplied, then 30 - 20 - 40 = 
24,000 multiplications are used to obtain the 30 x 40 matrix A, A,. Then, to multiply A, A, and 
A, requires 30 - 40 - 10 = 12,000 multiplications. Hence, a total of 

24,000 + 12,000 = 36,000 
multiplications are used. 


Clearly, the first method is more efficient. < 


We will return to this problem in Exercise 57 in Section 8.1. Algorithms for determining 
the most efficient way to carry out matrix-chain multiplication are discussed in [CoLeRiSt09]. 


EXAMPLE 10 
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3.3.4 Algorithmic Paradigms 


In Section 3.1 we introduced the basic notion of an algorithm. We provided examples of many 
different algorithms, including searching and sorting algorithms. We also introduced the con- 
cept of a greedy algorithm, giving examples of several problems that can be solved by greedy 
algorithms. Greedy algorithms provide an example of an algorithmic paradigm, that is, a gen- 
eral approach based on a particular concept that can be used to construct algorithms for solving 
a variety of problems. 

In this book we will construct algorithms for solving many different problems based on a 
variety of algorithmic paradigms, including the most widely used algorithmic paradigms. These 
paradigms can serve as the basis for constructing efficient algorithms for solving a wide range 
of problems. 

Some of the algorithms we have already studied are based on an algorithmic paradigm 
known as brute force, which we will describe in this section. Algorithmic paradigms, stud- 
ied later in this book, include divide-and-conquer algorithms studied in Chapter 8, dynamic 
programming, also studied in Chapter 8, backtracking, studied in Chapter 10, and probabilis- 
tic algorithms, studied in Chapter 7. There are many important algorithmic paradigms besides 
those described in this book. Consult books on algorithm design such as [K1Ta06] to learn more 
about them. 


BRUTE-FORCE ALGORITHMS Brute force is an important, and basic, algorithmic 
paradigm. In a brute-force algorithm, a problem is solved in the most straightforward manner 
based on the statement of the problem and the definitions of terms. Brute-force algorithms are 
designed to solve problems without regard to the computing resources required. For example, 
in some brute-force algorithms the solution to a problem is found by examining every possible 
solution, looking for the best possible. In general, brute-force algorithms are naive approaches 
for solving problems that do not take advantage of any special structure of the problem or clever 
ideas. 

Note that Algorithm 1 in Section 3.1 for finding the maximum number in a sequence is a 
brute-force algorithm because it examines each of the n numbers in a sequence to find the max- 
imum term. The algorithm for finding the sum of n numbers by adding one additional number 
at a time is also a brute-force algorithm, as is the algorithm for matrix multiplication based on 
its definition (Algorithm 1). The bubble, insertion, and selection sorts (described in Section 3.1 
in Algorithms 4 and 5 and in the preamble of Exercise 43, respectively) are also considered to 
be brute-force algorithms; all three of these sorting algorithms are straightforward approaches 
much less efficient than other sorting algorithms such as the merge sort and the quick sort dis- 
cussed in Chapters 5 and 8. 

Although brute-force algorithms are often inefficient, they are often quite useful. A brute- 
force algorithm may be able to solve practical instances of problems, particularly when the input 
is not too large, even if it is impractical to use this algorithm for larger inputs. Furthermore, 
when designing new algorithms to solve a problem, the goal is often to find a new algorithm 
that is more efficient than a brute-force algorithm. One such problem of this type is described 
in Example 10. 


Construct a brute-force algorithm for finding the closest pair of points in a set of n points in 
the plane and provide a worst-case big-O estimate for the number of bit operations used by the 
algorithm. 


Solution: Suppose that we are given as input the points (x,, y1), Œz 2), <--> Œp Yn). Recall that 
the distance between (x; y;) and (x Yj) is (i= x) + Q- y,;)*. A brute-force algorithm can 


find the closest pair of these points by computing the distances between all pairs of the n points 
and determining the smallest distance. (We can make one small simplification to make the com- 
putation easier; we can compute the square of the distance between pairs of points to find the 
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closest pair, rather than the distance between these points. We can do this because the square 
of the distance between a pair of points is smallest when the distance between these points is 
smallest.) 


ALGORITHM 3 Brute-Force Algorithm for Closest Pair of Points. 


procedure closest-pair((x, y1), (X2, Y2), ---, Xm Yn): pairs of real numbers) 
min= co 
for i := 2 ton 
for j := 1 toi— 1 
if (; -x + ©- y)? < min then 

min := (x; — x)? + = yD 

closest pair := ((x; yi), Œ; y) 
return closest pair 


To estimate the number of operations used by the algorithm, first note that there are 
n(n — 1)/2 pairs of points (x; y;), (x; y) that we loop through (as the reader should verify). 
For each such pair we compute (x, — x) + ©- y;)?, compare it with the current value of min, 
and if it is smaller than min, replace the current value of min by this new value. It follows that 
this algorithm uses @(n7) operations, in terms of arithmetic operations and comparisons. 

In Chapter 8 we will devise an algorithm that determines the closest pair of points when 
given n points in the plane as input that has O(n log n) worst-case complexity. The original dis- 
covery of such an algorithm, much more efficient than the brute-force approach, was considered 
quite surprising. < 


3.3.5 Understanding the Complexity of Algorithms 


Table 1 displays some common terminology used to describe the time complexity of algorithms. 
For example, an algorithm that finds the largest of the first 100 terms of a list of n elements by 
applying Algorithm 1 to the sequence of the first 100 terms, where n is an integer with n > 100, 
has constant complexity because it uses 99 comparisons no matter what n is (as the reader 
can verify). The linear search algorithm has linear (worst-case or average-case) complexity 
and the binary search algorithm has logarithmic (worst-case) complexity. Many important 
algorithms have n log n, or linearithmic (worst-case) complexity, such as the merge sort, which 
we will introduce in Chapter 4. (The word linearithmic is a combination of the words linear and 
logarithmic.) 


TABLE 1 Commonly Used Terminology for the 
Complexity of Algorithms. 


Complexity Terminology 


(1) Constant complexity 
O(log n) Logarithmic complexity 
O(n) Linear complexity 


O(n log n) Linearithmic complexity 


O(n?) Polynomial complexity 


©(b"), where b > 1 Exponential complexity 


O(n!) Factorial complexity 


Links » 
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An algorithm has polynomial complexity if it has complexity @(n”), where b is an inte- 
ger with b > 1. For example, the bubble sort algorithm is a polynomial-time algorithm because 
it uses O(n?) comparisons in the worst case. An algorithm has exponential complexity if it 
has time complexity @(b”), where b > 1. The algorithm that determines whether a compound 
proposition in n variables is satisfiable by checking all possible assignments of truth variables 
is an algorithm with exponential complexity, because it uses @(2”) operations. Finally, an al- 
gorithm has factorial complexity if it has O(n!) time complexity. The algorithm that finds all 
orders that a traveling salesperson could use to visit n cities has factorial complexity; we will 
discuss this algorithm in Chapter 9. 


TRACTABILITY A problem that is solvable using an algorithm with polynomial (or better) 
worst-case complexity is called tractable, because the expectation is that the algorithm will pro- 
duce the solution to the problem for reasonably sized input in a relatively short time. However, 
if the polynomial in the big-© estimate has high degree (such as degree 100) or if the coeffi- 
cients are extremely large, the algorithm may take an extremely long time to solve the problem. 
Consequently, that a problem can be solved using an algorithm with polynomial worst-case 
time complexity is no guarantee that the problem can be solved in a reasonable amount of time 
for even relatively small input values. Fortunately, in practice, the degree and coefficients of 
polynomials in such estimates are often small. 

The situation is much worse for problems that cannot be solved using an algorithm with 
worst-case polynomial time complexity. Such problems are called intractable. Usually, but not 
always, an extremely large amount of time is required to solve the problem for the worst cases 
of even small input values. In practice, however, there are situations where an algorithm with a 
certain worst-case time complexity may be able to solve a problem much more quickly for most 
cases than for its worst case. When we are willing to allow that some, perhaps small, number of 
cases may not be solved in a reasonable amount of time, the average-case time complexity is a 
better measure of how long an algorithm takes to solve a problem. Many problems important in 
industry are thought to be intractable but can be practically solved for essentially all sets of in- 
put that arise in daily life. Another way that intractable problems are handled when they arise in 
practical applications is that instead of looking for exact solutions of a problem, approximate so- 
lutions are sought. It may be the case that fast algorithms exist for finding such approximate solu- 
tions, perhaps even with a guarantee that they do not differ by very much from an exact solution. 

Some problems even exist for which it can be shown that no algorithm exists for solving 
them. Such problems are called unsolvable (as opposed to solvable problems that can be solved 
using an algorithm). The first proof that there are unsolvable problems was provided by the great 
English mathematician and computer scientist Alan Turing when he showed that the halting 
problem is unsolvable. Recall that we proved that the halting problem is unsolvable in Section 
3.1. (A biography of Alan Turing and a description of some of his other work can be found in 
Chapter 13.) 


P VERSUS NP The study of the complexity of algorithms goes far beyond what we can de- 
scribe here. Note, however, that many solvable problems are believed to have the property that 
no algorithm with polynomial worst-case time complexity solves them, but that a solution, if 
known, can be checked in polynomial time. Problems for which a solution can be checked in 
polynomial time are said to belong to the class NP (tractable problems are said to belong to 
class P). The abbreviation NP stands for nondeterministic polynomial time. The satisfiability 
problem, discussed in Section 1.3, is an example of an NP problem—we can quickly verify that 
an assignment of truth values to the variables of a compound proposition makes it true, but no 
polynomial time algorithm has been discovered for finding such an assignment of truth values. 
(For example, an exhaustive search of all possible truth values requires Q(2”) bit operations 
where n is the number of variables in the compound proposition.) 

There is also an important class of problems, called NP-complete problems, with the prop- 
erty that if any of these problems can be solved by a polynomial worst-case time algorithm, then 
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Courtesy of Dr. Stephen Cook 


all problems in the class NP can be solved by polynomial worst-case time algorithms. The sat- 
isfiability problem is also an example of an NP-complete problem. It is an NP problem and if 
a polynomial time algorithm for solving it were known, there would be polynomial time algo- 
rithms for all problems known to be in this class of problems (and there are many important 
problems in this class). This last statement follows from the fact that every problem in NP 
can be reduced in polynomial time to the satisfiability problem. Although more than 3000 NP- 
complete problems are now known, the satisfiability problem was the first problem shown to be 
NP-complete. The theorem that asserts this is known as the Cook-Levin theorem after Stephen 
Cook and Leonid Levin, who independently proved it in the early 1970s. 

The P versus NP problem asks whether NP, the class of problems for which it is possible 
to check solutions in polynomial time, equals P, the class of tractable problems. If P 4 NP, there 
would be some problems that cannot be solved in polynomial time, but whose solutions could 
be verified in polynomial time. The concept of NP-completeness is helpful in research aimed 
at solving the P versus NP problem, because NP-complete problems are the problems in NP 
considered most likely not to be in P, as every problem in NP can be reduced to an NP-complete 
problem in polynomial time. A large majority of theoretical computer scientists believe that 
P Æ NP, which would mean that no NP-complete problem can be solved in polynomial time. 
One reason for this belief is that despite extensive research, no one has succeeded in showing 
that P = NP. In particular, no one has been able to find an algorithm with worst-case polynomial 
time complexity that solves any NP-complete problem. The P versus NP problem is one of the 
most famous unsolved problems in the mathematical sciences (which include theoretical com- 
puter science). It is one of the seven famous Millennium Prize Problems, of which six remain 
unsolved. A prize of $1,000,000 is offered by the Clay Mathematics Institute for its solution. 

For more information about the complexity of algorithms, consult the references, including 
[CoLeRiSt09], for this section listed at the end of this book. (Also, for a more formal discussion 
of computational complexity in terms of Turing machines, see Section 13.5.) 


PRACTICAL CONSIDERATIONS Note that a big-© estimate of the time complexity of an 
algorithm expresses how the time required to solve the problem increases as the input grows 
in size. In practice, the best estimate (that is, with the smallest reference function) that can 
be shown is used. However, big-© estimates of time complexity cannot be directly translated 
into the actual amount of computer time used. One reason is that a big-© estimate f(n) is 
O(g(n)), where f(n) is the time complexity of an algorithm and g(n) is a reference function, 
means that C, g(n) < f(n) < C,g(n) when n > k, where C,, C}, and k are constants. So without 
knowing the constants C}, C,, and k in the inequality, this estimate cannot be used to determine 
a lower bound and an upper bound on the number of operations used in the worst case. As 
remarked before, the time required for an operation depends on the type of operation and the 


STEPHEN COOK (BORN 1939) Stephen Cook was born in Buffalo, where his father worked as an industrial 
chemist and taught university courses. His mother taught English courses in a community college. While in 
high school Cook developed an interest in electronics through his work with a famous local inventor noted for 
inventing the first implantable cardiac pacemaker. 

Cook was a mathematics major at the University of Michigan, graduating in 1961. He did graduate work 
at Harvard, receiving a master’s degree in 1962 and a Ph.D. in 1966. Cook was appointed an assistant professor 
in the Mathematics Department at the University of California, Berkeley, in 1966. He was not granted tenure 
there, possibly because the members of the Mathematics Department did not find his work on what is now 
considered to be one of the most important areas of theoretical computer science of sufficient interest. In 1970, 
he joined the University of Toronto as an assistant professor, holding a joint appointment in the Computer 
Science Department and the Mathematics Department. He has remained at the University of Toronto, where he 
was appointed a University Professor in 1985. 


Cook is considered to be one of the founders of computational complexity theory. His 1971 paper “The Complexity of Theorem 


Proving Procedures’ 


’ formalized the notions of NP-completeness and polynomial-time reduction, showed that NP-complete problems 


exist by showing that the satisfiability problem is such a problem, and introduced the notorious P versus NP problem. 
Cook has received many awards, including the 1982 Turing Award. He is married and has two sons. Among his interests are 
playing the violin and racing sailboats. 
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TABLE 2 The Computer Time Used by Algorithms. 


Problem Size Bit Operations Used 


3x 107! s 107° 1078 s 
7x107! s s os ney 4x10! yr 

1.0107! s ‘ s 1075 s * 

1.3 x 107" s ‘ 65 1073 s * 

1.7 x 107! s ‘ 55 0.1s * 
2x107" s ‘ s 0.17 min * 


computer being used. Often, instead of a big-© estimate on the worst-case time complexity of 
an algorithm, we have only a big-O estimate. Note that a big-O estimate on the time complexity 
of an algorithm provides an upper, but not a lower, bound on the worst-case time required for 
the algorithm as a function of the input size. Nevertheless, for simplicity, we will often use 
big-O estimates when describing the time complexity of algorithms, with the understanding 
that big-© estimates would provide more information. 

Table 2 displays the time needed to solve problems of various sizes with an algorithm using 
the indicated number n of bit operations, assuming that each bit operation takes 10~'! seconds, a 
reasonable estimate of the time required for a bit operation using the fastest computers available 
in 2018. Times of more than 10!°° years are indicated with an asterisk. In the future, these times 
will decrease as faster computers are developed. We can use the times shown in Table 2 to see 
whether it is reasonable to expect a solution to a problem of a specified size using an algorithm 
with known worst-case time complexity when we run this algorithm on a modern computer. 
Note that we cannot determine the exact time a computer uses to solve a problem with input of 
a particular size because of a myriad of issues involving computer hardware and the particular 
software implementation of the algorithm. 

It is important to have a reasonable estimate for how long it will take a computer to solve a 
problem. For instance, if an algorithm requires approximately 10 hours, it may be worthwhile 
to spend the computer time (and money) required to solve this problem. But, if an algorithm 
requires approximately 10 billion years to solve a problem, it would be unreasonable to use re- 
sources to implement this algorithm. One of the most interesting phenomena of modern technol- 
ogy is the tremendous increase in the speed and memory space of computers. Another important 
factor that decreases the time needed to solve problems on computers is parallel processing, 
which is the technique of performing sequences of operations simultaneously. 

Efficient algorithms, including most algorithms with polynomial time complexity, benefit 
most from significant technology improvements. However, these technology improvements 
offer little help in overcoming the complexity of algorithms of exponential or factorial time 
complexity. Because of the increased speed of computation, increases in computer memory, and 
the use of algorithms that take advantage of parallel processing, many problems that were con- 
sidered impossible to solve five years ago are now routinely solved, and certainly five years from 
now this statement will still be true. This is even true when the algorithms used are intractable. 


Exercises 
1. Give a big-O estimate for the number of operations 2. Give a big-O estimate for the number additions used in 
(where an operation is an addition or a multiplication) this segment of an algorithm. 
used in this segment of an algorithm. t:=0 
t:=0 fori := 1 ton 
for i := 1 to3 forj := 1 ton 
for j := 1 to 4 t:=t+i+j 


t:=t+ij 
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. Give a big-O estimate for the number of operations, 


where an operation is a comparison or a multiplication, 
used in this segment of an algorithm (ignoring compar- 
isons used to test the conditions in the for loops, where 
4, Ay, ..., A, are positive real numbers). 

m:=0 

fori :=1ton 

forj:=i+l1ton 
m := max(a;a;, m) 


. Give a big-O estimate for the number of operations, 


where an operation is an addition or a multiplication, 
used in this segment of an algorithm (ignoring compar- 
isons used to test the conditions in the while loop). 


i:=1 

t:=0 

while i <n 
t:=t+i 
i:= 2i 


. How many comparisons are used by the algorithm given 


in Exercise 16 of Section 3.1 to find the smallest natural 
number in a sequence of n natural numbers? 


. a) Use pseudocode to describe the algorithm that puts 


the first four terms of a list of real numbers of arbitrary 
length in increasing order using the insertion sort. 

b) Show that this algorithm has time complexity O(1) in 
terms of the number of comparisons used. 


. Suppose that an element is known to be among the first 


four elements in a list of 32 elements. Would a lin- 
ear search or a binary search locate this element more 
rapidly? 


. Given a real number x and a positive integer k, determine 


the number of multiplications used to find x~” starting 
with x and successively squaring (to find x’, x*, and so 
on). Is this a more efficient way to find x” than by multi- 
plying x by itself the appropriate number of times? 


. Give a big-O estimate for the number of comparisons 


used by the algorithm that determines the number of 1s 
in a bit string by examining each bit of the string to deter- 
mine whether it is a 1 bit (see Exercise 25 of Section 3.1). 


a) Show that this algorithm determines the number of 1 
bits in the bit string S: 


procedure bit count(S: bit string) 


count := 0 

while S 4 0 
count := count + 1 
S:=SA(S—1) 


return count {count is the number of 1s in S} 


Here S — 1 is the bit string obtained by changing the 
rightmost 1 bit of S to a O and all the O bits to the 
right of this to 1s. [Recall that S A (S — 1) is the bit- 
wise AND of S and S — 1.] 

b) How many bitwise AND operations are needed to find 
the number of | bits in a string S using the algorithm 
in part (a)? 


11. 


12. 


13. 


14. 


a) Suppose we have n subsets S),5S5,...,5,, of the set 
{1, 2, ..., n}. Express a brute-force algorithm that de- 
termines whether there is a disjoint pair of these sub- 
sets. [Hint: The algorithm should loop through the 
subsets; for each subset S,, it should then loop through 
all other subsets; and for each of these other subsets 
S;, it should loop through all elements k in S; to deter- 
mine whether k also belongs to S;.] 


b) Give a big-O estimate for the number of times the al- 
gorithm needs to determine whether an integer is in 
one of the subsets. 


Consider the following algorithm, which takes as input a 
sequence of n integers a), do, ..., a, and produces as out- 
put a matrix M = {mi} where Mm is the minimum term 
in the sequence of integers a,, a;,,, -oaj for j > i and 
m, = 0 otherwise. 
initialize M so that m, = a; if j => i and m; =0 
otherwise 
for i := 1 ton 
forj:=i+l1ton 
fork :=i+1toj 
my := min(m; ay) 
return M= {m,} {m; is the minimum term of 
a; Aip -> Aj} 
a) Show that this algorithm uses O(n?) comparisons to 
compute the matrix M. 


b) Show that this algorithm uses Q(n*) comparisons to 
compute the matrix M. Using this fact and part (a), 
conclude that the algorithms uses O(n?) comparisons. 
[Hint: Only consider the cases where i < n/4 and 
j = 3n/4 in the two outer loops in the algorithm.] 

The conventional algorithm for evaluating a polyno- 

mial a,x" +a,_,x"|++-+a,;x+d) at x=c can be 

expressed in pseudocode by 


procedure polynomial(c, dg, a, ...,a,: teal numbers) 
power := 1 
Y := do 


for i := 1 ton 
power := power * C 
y := y + a; * power 
return y {y=a,c" + a,c"! +- +ajc +ap} 


where the final value of y is the value of the polynomial 

atx=c. 

a) Evaluate 3x? +x+ 1 at x=2 by working through 
each step of the algorithm showing the values as- 
signed at each assignment step. 


b) Exactly how many multiplications and additions are 
used to evaluate a polynomial of degree n at x = c? 
(Do not count additions used to increment the loop 
variable.) 

There is a more efficient algorithm (in terms of the 

number of multiplications and additions used) for eval- 

uating polynomials than the conventional algorithm de- 
scribed in the previous exercise. It is called Horner’s 
method. This pseudocode shows how to use this method 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


to find the value of a,x” +a,1x"! +- +a;x + ao 
atx =c. 


procedure Horner(c, dg, 44, dy, ..., @,: real numbers) 
yi=a, 
for i:= 1 ton 
yr=y*ecta,; 
return y {y =a,c" +a,_\c" | +++ + ac + do} 


a) Evaluate 3x? +x+ 1 at x=2 by working through 
each step of the algorithm showing the values as- 
signed at each assignment step. 

b) Exactly how many multiplications and additions are 
used by this algorithm to evaluate a polynomial of 
degree n at x = c? (Do not count additions used to 
increment the loop variable.) 

What is the largest n for which one can solve within one 

second a problem using an algorithm that requires f(n) 

bit operations, where each bit operation is carried out in 

10~° seconds, with these functions f(n)? 

a) logn b) n c) nlogn 

d) n e) 2" f) n! 

What is the largest n for which one can solve within a 

day using an algorithm that requires f(n) bit operations, 

where each bit operation is carried out in 107!! seconds, 

with these functions f(n)? 


a) logn b) 1000n o) n? 
d) 10007? e) n? f) 2” 
g) 92n h) 22" 


What is the largest n for which one can solve within 
a minute using an algorithm that requires f(n) bit op- 
erations, where each bit operation is carried out in 
107!? seconds, with these functions f(n)? 


a) loglogn b) logn c) (logn)? 
d) 1,000,000n e) n? f) 2” 
g) 2" 


How much time does an algorithm take to solve a prob- 
lem of size n if this algorithm uses 2n? + 2” operations, 
each requiring 107° seconds, with these values of n? 

a) 10 b) 20 c) 50 d) 100 


How much time does an algorithm using 2°° operations 
need if each operation takes these amounts of time? 

a) 10% s b) 10° s e) 10° s 

What is the effect in the time required to solve a prob- 
lem when you double the size of the input from n to 
2n, assuming that the number of milliseconds the algo- 
rithm uses to solve the problem with input size n is each 
of these functions? [Express your answer in the simplest 
form possible, either as a ratio or a difference. Your an- 
swer may be a function of n or a constant.] 


a) loglogn b) logn c) 1007 
d) nlogn e) n? f) w? 
g) 2" 


What is the effect in the time required to solve a prob- 
lem when you increase the size of the input from n to 
n + 1, assuming that the number of milliseconds the al- 
gorithm uses to solve the problem with input size n is 


22. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 
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each of these functions? [Express your answer in the sim- 
plest form possible, either as a ratio or a difference. Your 
answer may be a function of n or a constant. ] 


a) logn b) 1007 9n 
d) n? e) 2” f) 2”? 
g) n! 


Determine the least number of comparisons, or best-case 

performance, 

a) required to find the maximum of a sequence of n in- 
tegers, using Algorithm 1 of Section 3.1. 

b) used to locate an element in a list of n terms with a 
linear search. 

c) used to locate an element in a list of n terms using a 
binary search. 

Analyze the average-case performance of the linear 

search algorithm, if exactly half the time the element x 

is not in the list, and if x is in the list, it is equally likely 

to be in any position. 

An algorithm is called optimal for the solution of a prob- 

lem with respect to a specified operation if there is no al- 

gorithm for solving this problem using fewer operations. 

a) Show that Algorithm 1 in Section 3.1 is an optimal 
algorithm with respect to the number of comparisons 
of integers. [Note: Comparisons used for bookkeep- 
ing in the loop are not of concern here.] 

b) Is the linear search algorithm optimal with respect to 
the number of comparisons of integers (not including 
comparisons used for bookkeeping in the loop)? 

Describe the worst-case time complexity, measured in 

terms of comparisons, of the ternary search algorithm de- 

scribed in Exercise 27 of Section 3.1. 


Describe the worst-case time complexity, measured in 
terms of comparisons, of the search algorithm described 
in Exercise 28 of Section 3.1. 


Analyze the worst-case time complexity of the algorithm 
you devised in Exercise 29 of Section 3.1 for locating a 
mode in a list of nondecreasing integers. 

Analyze the worst-case time complexity of the algorithm 
you devised in Exercise 30 of Section 3.1 for locating all 
modes in a list of nondecreasing integers. 


Analyze the worst-case time complexity of the algorithm 
you devised in Exercise 33 of Section 3.1 for finding the 
first term of a sequence of integers equal to some previ- 
ous term. 


Analyze the worst-case time complexity of the algorithm 
you devised in Exercise 34 of Section 3.1 for finding all 
terms of a sequence that are greater than the sum of all 
previous terms. 

Analyze the worst-case time complexity of the algorithm 
you devised in Exercise 35 of Section 3.1 for finding the 
first term of a sequence less than the immediately preced- 
ing term. 

Determine the worst-case complexity in terms of com- 
parisons of the algorithm from Exercise 5 in Section 3.1 
for determining all values that occur more than once in a 
sorted list of integers. 
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33. Determine the worst-case complexity in terms of com- 
parisons of the algorithm from Exercise 9 in Section 3.1 
for determining whether a string of n characters is a palin- 
drome. 


34. How many comparisons does the selection sort (see 
preamble to Exercise 43 in Section 3.1) use to sort n 
items? Use your answer to give a big-O estimate of the 
complexity of the selection sort in terms of number of 
comparisons for the selection sort. 


35. Determine a big-O estimate for the worst-case complex- 
ity in terms of number of comparisons used and the 
number of terms swapped by the binary insertion sort de- 
scribed in the preamble to Exercise 49 in Section 3.1. 


36. Determine the number of character comparisons used by 
the naive string matcher to look for a pattern of m char- 
acters in a text with n characters if the first character of 
the pattern does not occur in the text. 


37. Determine a big-O estimate of the number of character 
comparisons used by the naive string matcher to find all 
occurrences of a pattern of m characters in a text with n 
characters, in terms of the parameters m and n. 


38. Determine big-O estimates for the algorithms for decid- 
ing whether two strings are anagrams from parts (a) and 
(b) of Exercise 31 of Section 3.1. 


39. Determine big-O estimates for the algorithms for finding 
the closest of n real numbers from parts (a) and (b) of 
Exercise 32 of Section 3.1. 


40. Show that the greedy algorithm for making change for n 
cents using quarters, dimes, nickels, and pennies has O(n) 
complexity measured in terms of comparisons needed. 

Exercises 41 and 42 deal with the problem of scheduling the 

most talks possible given the start and end times of n talks. 


41. Find the complexity of a brute-force algorithm for 
scheduling the talks by examining all possible subsets of 
the talks. [Hint: Use the fact that a set with n elements 
has 2” subsets. | 


42. Find the complexity of the greedy algorithm for schedul- 
ing the most talks by adding at each step the talk with the 


Key Terms and Results 


TERMS 


algorithm: a finite sequence of precise instructions for per- 
forming a computation or solving a problem 


searching algorithm: the problem of locating an element in a 
list 

linear search algorithm: a procedure for searching a list ele- 
ment by element 


binary search algorithm: a procedure for searching an or- 
dered list by successively splitting the list in half 


sorting: the reordering of the elements of a list into prescribed 
order 


earliest end time compatible with those already sched- 
uled (Algorithm 7 in Section 3.1). Assume that the 
talks are not already sorted by earliest end time and as- 
sume that the worst-case time complexity of sorting is 
O(n log n). 

43. Describe how the number of comparisons used in the 
worst case changes when these algorithms are used to 
search for an element of a list when the size of the list 
doubles from n to 2n, where n is a positive integer. 


a) linear search b) binary search 
44. Describe how the number of comparisons used in the 
worst case changes when the size of the list to be sorted 
doubles from n to 2n, where n is a positive integer when 
these sorting algorithms are used. 
a) bubble sort b) insertion sort 
c) selection sort (described in the preamble to Exer- 
cise 43 in Section 3.1) 
d) binary insertion sort (described in the preamble to Ex- 
ercise 49 in Section 3.1) 


An n Xn matrix is called upper triangular if a, = 0 when- 

ever i >j. 

45. From the definition of the matrix product, describe an al- 
gorithm in English for computing the product of two up- 
per triangular matrices that ignores those products in the 
computation that are automatically equal to zero. 

46. Give a pseudocode description of the algorithm in Exer- 
cise 45 for multiplying two upper triangular matrices. 

47. How many multiplications of entries are used by the al- 
gorithm found in Exercise 45 for multiplying two n xn 
upper triangular matrices? 

In Exercises 48—49 assume that the number of multiplications 

of entries used to multiply a p X q matrix and a q X r matrix 

is pqr. 

48. What is the best order to form the product ABC if A, 
B, and C are matrices with dimensions 3 x 9, 9 x 4, and 
4 x 2, respectively? 

49. What is the best order to form the product ABCD if A, B, 
C, and D are matrices with dimensions 30 x 10, 10 x 40, 
40 x 50, and 50 x 30, respectively? 


string searching: given a string, determining all the occur- 
rences where this string occurs within a longer string 


F(x) is O(g(x)): the fact that [f(x)| < Clg(x)| for all x > k for 
some constants C and k 


witness to the relationship f(x) is O(g(x)): a pair C and k such 
that |f(x)| < Clg(x)| whenever x > k 


fœ) is Q(g(x)): the fact that [f] > Clg(x)| for all x > k for 
some positive constants C and k 


f(x) is O(g(x)): the fact that f(x) is both O(g(x)) and Q(g(x)) 


time complexity: the amount of time required for an algorithm 
to solve a problem 


space complexity: the amount of space in computer memory 
required for an algorithm to solve a problem 

worst-case time complexity: the greatest amount of time re- 
quired for an algorithm to solve a problem of a given size 

average-case time complexity: the average amount of time 
required for an algorithm to solve a problem of a given size 

algorithmic paradigm: a general approach for constructing 
algorithms based on a particular concept 

brute force: the algorithmic paradigm based on constructing 
algorithms for solving problems in a naive manner from the 
statement of the problem and definitions 

greedy algorithm: an algorithm that makes the best choice at 
each step according to some specified condition 

tractable problem: a problem for which there is a worst-case 
polynomial-time algorithm that solves it 

intractable problem: a problem for which no worst-case 
polynomial-time algorithm exists for solving it 

solvable problem: a problem that can be solved by an algo- 
rithm 

unsolvable problem: a problem that cannot be solved by an 
algorithm 


Review Questions 


1. a) Define the term algorithm. 
b) What are the different ways to describe algorithms? 


c) What is the difference between an algorithm for solv- 
ing a problem and a computer program that solves this 
problem? 


2. a) Describe, using English, an algorithm for finding the 
largest integer in a list of n integers. 


b) Express this algorithm in pseudocode. 


c) How many comparisons does the algorithm use? 


3. a) State the definition of the fact that f(n) is O(g(n)), 
where f(n) and g(n) are functions from the set of pos- 
itive integers to the set of real numbers. 


b) Use the definition of the fact that f(n) is O(g(n)) 
directly to prove or disprove that n? + 18 + 107 is 
O(n?). 

c) Use the definition of the fact that f(n) is 
O(g(n)) directly to prove or disprove that n> is 
O(n? + 18n + 107). 


4. List these functions so that each function is big-O of 
the next function in the list: (log n}, n? /1,000,000, yn, 
1007 + 101, 3”, n!, 2”n?. 


5. a) How can you produce a big-O estimate for a function 
that is the sum of different terms where each term is 
the product of several functions? 


b) Give a big-O estimate for the function f(n) = 
(n!+ 1)(2" + 1) + (n"? + 8n"3)(n3 +2”). For the 
function g in your estimate f(x) is O(g(x)), use a sim- 
ple function of smallest possible order. 
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RESULTS 


linear and binary search algorithms: (given in Section 3.1) 


bubble sort: a sorting that uses passes where successive items 
are interchanged if they are in the wrong order 


insertion sort: a sorting that at the jth step inserts the jth ele- 
ment into the correct position in the list, when the first j — 1 
elements of the list are already sorted 


The linear search has O(n) worst case time complexity. 

The binary search has O(log n) worst case time complexity. 

The bubble and insertion sorts have O(n?) worst case time 
complexity. 

logn! is O(n log n). 


If fŒ) is O) and f,(x) is O8), then (f, + 
is O(max(g (x), 8(x))) and (F2) is O81820). 


If dg, a}, ..., a, are real numbers with a, #0, then a,x” + 
A,X"! ++ +ax +a is ©”), and hence O(n) and 
Q(n). 


6. a) Define what the worst-case time complexity, average- 
case time complexity, and best-case time complexity 
(in terms of comparisons) mean for an algorithm that 
finds the smallest integer in a list of n integers. 

b) What are the worst-case, average-case, and best-case 
time complexities, in terms of comparisons, of the al- 
gorithm that finds the smallest integer in a list of n 
integers by comparing each of the integers with the 
smallest integer found so far? 

7. a) Describe the linear search and binary search algo- 
rithm for finding an integer in a list of integers in in- 
creasing order. 

b) Compare the worst-case time complexities of these 
two algorithms. 

c) Is one of these algorithms always faster than the other 
(measured in terms of comparisons)? 

8. a) Describe the bubble sort algorithm. 

b) Use the bubble sort algorithm to sort the list 5, 2, 4, 
1,3. 

c) Give a big-O estimate for the number of comparisons 
used by the bubble sort. 

9. a) Describe the insertion sort algorithm. 

b) Use the insertion sort algorithm to sort the list 2, 5, 1, 
4, 3. 

c) Give a big-O estimate for the number of comparisons 
used by the insertion sort. 

10. a) Explain the concept of a greedy algorithm. 

b) Provide an example of a greedy algorithm that pro- 
duces an optimal solution and explain why it produces 
an optimal solution. 
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c) Provide an example of a greedy algorithm that does 
not always produce an optimal solution and explain 
why it fails to do so. 


Supplementary Exercises 


1. 


*7, 


a) Describe an algorithm for locating the last occurrence 
of the largest number in a list of integers. 
b) Estimate the number of comparisons used. 


. a) Describe an algorithm for finding the first and second 


largest elements in a list of integers. 
b) Estimate the number of comparisons used. 


. a) Give an algorithm to determine whether a bit string 


contains a pair of consecutive zeros. 
b) How many comparisons does the algorithm use? 


. a) Suppose that a list contains integers that are in order 


of largest to smallest and an integer can appear repeat- 
edly in this list. Devise an algorithm that locates all 
occurrences of an integer x in the list. 

b) Estimate the number of comparisons used. 


. a) Adapt Algorithm 1 in Section 3.1 to find the maxi- 


mum and the minimum of a sequence of n elements 
by employing a temporary maximum and a temporary 
minimum that is updated as each successive element 
is examined. 

b) Describe the algorithm from part (a) in pseudocode. 

c) How many comparisons of elements in the sequence 
are carried out by this algorithm? (Do not count com- 
parisons used to determine whether the end of the se- 
quence has been reached.) 


. a) Describe in detail (and in English) the steps of an al- 


gorithm that finds the maximum and minimum of a 
sequence of n elements by examining pairs of suc- 
cessive elements, keeping track of a temporary maxi- 
mum and a temporary minimum. If n is odd, both the 
temporary maximum and temporary minimum should 
initially equal the first term, and if n is even, the tem- 
porary minimum and temporary maximum should be 
found by comparing the initial two elements. The tem- 
porary maximum and temporary minimum should be 
updated by comparing them with the maximum and 
minimum of the pair of elements being examined. 

b) Express the algorithm described in part (a) in pseu- 
docode. 

c) How many comparisons of elements of the sequence 
are carried out by this algorithm? (Do not count com- 
parisons used to determine whether the end of the se- 
quence has been reached.) How does this compare to 
the number of comparisons used by the algorithm in 
Exercise 5? 

Show that the worst-case complexity in terms of compar- 

isons of an algorithm that finds the maximum and mini- 

mum of n elements is at least [3/2] — 2. 


. Devise an efficient algorithm for finding the second 


largest element in a sequence of n elements and deter- 
mine the worst-case complexity of your algorithm. 


11. 


10. 


Define what it means for a problem to be tractable and 
what it means for a problem to be solvable. 


. Devise an algorithm that finds all equal pairs of sums of 


two terms of a sequence of n numbers, and determine the 
worst-case complexity of your algorithm. 


Devise an algorithm that finds the closest pair of integers 
in a sequence of n integers, and determine the worst-case 
complexity of your algorithm. [Hint: Sort the sequence. 
Use the fact that sorting can be done with worst-case time 
complexity O(n log n).] 


The shaker sort (or bidirectional bubble sort) successively 
compares pairs of adjacent elements, exchanging them if they 
are out of order, and alternately passing through the list from 
the beginning to the end and then from the end to the begin- 
ning until no exchanges are needed. 


11. 


12. 
13. 


14. 


15. 
16. 
17. 
18. 
*19, 
* 20. 
21. 


22. 


23. 
24. 
*25. 


*26. 


*27, 


Show the steps used by the shaker sort to sort the list 3, 
5, 1, 4, 6, 2. 

Express the shaker sort in pseudocode. 

Show that the shaker sort has O(n”) complexity measured 
in terms of the number of comparisons it uses. 

Explain why the shaker sort is efficient for sorting lists 
that are already in close to the correct order. 

Show that (n logn + n?Y is O(n°). 

Show that 8x? + 12x + 100logx is O(). 

Give a big-O estimate for (2 + x(log x)*) - (2* + x3). 
Find a big-O estimate for }/_ j(j+ 1). 

Show that n! is not O(2”). 

Show that n” is not O(n!). 

Find all pairs of functions of the same order in this list 
of functions: n? + (log ny, n° +n, n+ log2” +1, (a+ 
1} —(n— 1°, and (n + log ny. 

Find all pairs of functions of the same order in this list 
of functions n? + 2”, n? + 2! n? +22" n? 4+ n!, n? + 3”, 
and (n? + 1). 

Find an integer n with n > 2 for which n>” <2”, 

Find an integer n with n > 2 for which (log ne < yn. 
Arrange the functions n”, (logn)?, n!®%!, (1.0001)", 
2V!e", and n(logn)!! in a list so that each function is 
big-O of the next function. [Hint: To determine the rela- 
tive size of some of these functions, take logarithms. ] 
Arrange the function 219, 2°, 27!) 2%, plosn, 
nlognloglogn, n>/*, nlogn)*/?, and n*/3(logn)* in a 
list so that each function is big-O of the next function. 
[Hint: To determine the relative size of some of these 
functions, take logarithms. ] 


Give an example of two increasing functions f (n) and g(n) 
from the set of positive integers to the set of positive inte- 
gers such that neither f(n) is O(g(”)) nor g(n) is O(f(n)). 


28. Show that if the denominations of coins are 0°, c!, ..., ck, 
where k is a positive integer and c is a positive integer, 
c > I, the greedy algorithm always produces change us- 


ing the fewest coins possible. 


29. a) Use pseudocode to specify a brute-force algorithm 
that determines when given as input a sequence of n 
positive integers whether there are two distinct terms 
of the sequence that have as sum a third term. The al- 
gorithm should loop through all triples of terms of the 
sequence, checking whether the sum of the first two 
terms equals the third. 

b) Give a big-O estimate for the complexity of the brute- 
force algorithm from part (a). 


30. a) Devise a more efficient algorithm for solving the prob- 
lem described in Exercise 29 that first sorts the in- 
put sequence and then checks for each pair of terms 
whether their difference is in the sequence. 

b) Give a big-O estimate for the complexity of this al- 
gorithm. Is it more efficient than the brute-force algo- 
rithm from Exercise 29? 


Suppose we have s men and s women each with their prefer- 
ence lists for the members of the opposite gender, as described 
in the preamble to Exercise 64 in Section 3.1. We say that a 
woman w is a valid partner for a man m if there is some sta- 
ble matching in which they are paired. Similarly, a man m is a 
valid partner for a woman w if there is some stable matching 
in which they are paired. A matching in which each man is as- 
signed his valid partner ranking highest on his preference list 
is called male optimal, and a matching in which each woman 
is assigned her valid partner ranking lowest on her preference 
list is called female pessimal. 


31. Find all valid partners for each man and each woman if 
there are three men m, m,, and m, and three women w}, 
w, w; With these preference rankings of the men for the 
women, from highest to lowest: m,: w3, W1, W2; Mz! W3, 
W2, W1; M3: Wy, W3, W,; and with these preference rank- 
ings of the women for the men, from highest to lowest: 
W123, My, M13 W2: My, Mz, M3; W3: M3, My, M). 
* 32. Show that the deferred acceptance algorithm given in the 
preamble to Exercise 65 of Section 3.1, always produces 
a male optimal and female pessimal matching. 


33. Define what it means for a matching to be female optimal 
and for a matching to be male pessimal. 


* 34. Show that when woman do the proposing in the deferred 
acceptance algorithm, the matching produced is female 
optimal and male pessimal. 

In Exercises 35 and 36 we consider variations on the problem 
of finding stable matchings of men and women described in 
the preamble to Exercise 65 in Section 3.1. 


* 35. In this exercise we consider matching problems where 
there may be different numbers of men and women, so 
that it is impossible to match everyone with a member of 
the opposite gender. 

a) Extend the definition of a stable matching from that 
given in the preamble to Exercise 64 in Section 3.1 
to cover the case where there are unequal numbers of 
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men and women. Avoid all cases where a man and a 
woman would prefer each other to their current sit- 
uation, including those involving unmatched people. 
(Assume that an unmatched person prefers a match 
with a member of the opposite gender to remaining 
unmatched.) 


b) Adapt the deferred acceptance algorithm to find sta- 
ble matchings, using the definition of stable match- 
ings from part (a), when there are different numbers 
of men and women. 


c) Prove that all matchings produced by the algorithm 
from part (b) are stable, according to the definition 
from part (a). 
*36. In this exercise we consider matching problems where 
some man-woman pairs are not allowed. 


a) Extend the definition of a stable matching to cover the 
situation where there are the same number of men and 
women, but certain pairs of men and women are for- 
bidden. Avoid all cases where a man and a woman 
would prefer each other to their current situation, in- 
cluding those involving unmatched people. 


b) Adapt the deferred acceptance algorithm to find sta- 
ble matchings when there are the same number of men 
and women, but certain man-woman pairs are forbid- 
den. Be sure to consider people who are unmatched at 
the end of the algorithm. (Assume that an unmatched 
person prefers a match with a member of the opposite 
gender who is not a forbidden partner to remaining 
unmatched.) 


c) Prove that all matchings produced by the algorithm 
from (b) are stable, according to the definition in 

part (a). 
Exercises 37-40 deal with the problem of scheduling n jobs 
on a single processor. To complete job j, the processor must 
run job j for time t, without interruption. Each job has a dead- 
line d;. If we start job j at time s;, it will be completed at 
time e; = s; + t;. The lateness of the job measures how long 
it finishes after its deadline, that is, the lateness of job j is 
max(0, e; — d;). We wish to devise a greedy algorithm that 
minimizes the maximum lateness of a job among the n jobs. 


37. Suppose we have five jobs with specified required times 
and deadlines: t, = 25, d; = 50; t, = 15, d, = 60; t 
20, d; = 60; t, = 5, d, = 55; t; = 10, d; = 75. Find the 
maximum lateness of any job when the jobs are scheduled 
in this order (and they start at time 0): Job 3, Job 1, Job 4, 
Job 2, Job 5. Answer the same question for the schedule 
Job 5, Job 4, Job 3, Job 1, Job 2. 


38. The slackness of a job requiring time ¢ and with dead- 
line d is d — t, the difference between its deadline and the 
time it requires. Find an example that shows that schedul- 
ing jobs by increasing slackness does not always yield a 
schedule with the smallest possible maximum lateness. 


39. Find an example that shows that scheduling jobs in or- 
der of increasing time required does not always yield a 
schedule with the smallest possible maximum lateness. 
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* 40. Prove that scheduling jobs in order of increasing dead- 
lines always produces a schedule that minimizes the 
maximum lateness of a job. [Hint: First show that for a 
schedule to be optimal, jobs must be scheduled with no 
idle time between them and so that no job is scheduled 
before another with an earlier deadline. ] 

41. Suppose that we have a knapsack with total capac- 
ity of W kg. We also have n items where item j has 
mass w;. The knapsack problem asks for a subset of 
these n items with the largest possible total mass not 
exceeding W. 

a) Devise a brute-force algorithm for solving the knap- 
sack problem. 

b) Solve the knapsack problem when the capacity of 
the knapsack is 18 kg and there are five items: a 
5-kg sleeping bag, an 8-kg tent, a 7-kg food pack, 
a 4-kg container of water, and an 11-kg portable 
stove. 


In Exercises 42—46 we will study the problem of load balanc- 
ing. The input to the problem is a collection of p processors 
and n jobs, t, is the time required to run job j, jobs run without 
interruption on a single machine until finished, and a proces- 
sor can run only one job at a time. The load L, of processor 
k is the sum over all jobs assigned to processor k of the times 
required to run these jobs. The makespan is the maximum 
load over all the p processors. The load balancing problem 
asks for an assignment of jobs to processors to minimize the 
makespan. 


Computer Projects 
Write programs with these inputs and outputs. 


1. Given a list of n integers, find the largest integer in the list. 


2. Given a list of n integers, find the first and last occur- 
rences of the largest integer in the list. 


3. Given a list of n distinct integers, determine the position 
of an integer in the list using a linear search. 


4. Given an ordered list of n distinct integers, determine the 
position of an integer in the list using a binary search. 


5. Given a list of n integers, sort them using a bubble sort. 
6. Given a list of n integers, sort them using an insertion sort. 


7. Given two strings of characters use the naive string 
matching algorithm to determine whether the shorter 
string occurs in the longer string. 


Computations and Explorations 


42. Suppose we have three processors and five jobs requir- 
ing times t; = 3, h = 5, t, = 4, t, = 7, and t; = 8. Solve 
the load balancing problem for this input by finding the 
assignment of the five jobs to the three processors that 
minimizes the makespan. 


43. Suppose that L* is the minimum makespan when p pro- 
cessors are given n jobs, where t, is the time required to 
run job j. 

a) Show that L* > maxj_19 nf 


ee nj’ 


* 1 n 
b) Show that L* > 7 pe 


44. Write out in pseudocode the greedy algorithm that goes 
through the jobs in order and assigns each job to the pro- 
cessor with the smallest load at that point in the algo- 
rithm. 

45. Run the algorithm from Exercise 44 on the input given in 
Exercise 42. 

An approximation algorithm for an optimization problem 

produces a solution guaranteed to be close to an optimal so- 

lution. More precisely, suppose that the optimization problem 
asks for an input S that minimizes F(X) where F is some func- 

tion of the input X. If an algorithm always finds an input T 

with F(T) < cF(S), where c is a fixed positive real number, 

the algorithm is called a c-approximation algorithm for the 
problem. 
*46. Prove that the algorithm from Exercise 44 is a 2- 
approximation algorithm for the load balancing problem. 
[Hint: Use both parts of Exercise 43.] 


8. Given an integer n, use the cashier’s algorithm to find 
the change for n cents using quarters, dimes, nickels, and 
pennies. 

9. Given the starting and ending times of n talks, use the 
appropriate greedy algorithm to schedule the most talks 
possible in a single lecture hall. 

10. Given an ordered list of n integers and an integer x in the 
list, find the number of comparisons used to determine 
the position of x in the list using a linear search and using 
a binary search. 

11. Given a list of integers, determine the number of compar- 
isons used by the bubble sort and by the insertion sort to 
sort this list. 


Use a computational program or programs you have written to do these exercises. 


1. We know that n? is O(d") when b and d are positive num- 
bers with d > 2. Give values of the constants C and k such 


that n? < Cd" whenever x > k for each of these sets of val- 
ues: b = 10, d = 2; b = 20, d = 3; b = 1000, d = 7. 


2. Compute the change for different values of n with coins of 
different denominations using the cashier’s algorithm and 
determine whether the smallest number of coins was used. 
Can you find conditions so that the cashier’s algorithm is 
guaranteed to use the fewest coins possible? 

3. Using a generator of random orderings of the integers 
1,2, ..., n, find the number of comparisons used by the 
bubble sort, insertion sort, binary insertion sort, and 
selection sort to sort these integers. 


Writing Projects 
Respond to these with essays using outside sources. 


1. Examine the history of the word algorithm and describe 
the use of this word in early writings. 

2. Look up Bachmann’s original introduction of big-O no- 
tation. Explain how he and others have used this notation. 

3. Explain how sorting algorithms can be classified into a 
taxonomy based on the underlying principle on which 
they are based. 

4. Describe the radix sort algorithm. 

5. Describe some of the different algorithms for string 
matching. 

6. Describe some of the different applications of string 
matching in bioinformatics. 

7. Describe the historic trends in how quickly processors 
can perform operations and use these trends to estimate 
how quickly processors will be able to perform operations 
in the next 20 years. 


4. 


*5, 


10. 


11. 


12. 
13. 
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Collect experimental evidence comparing the number of 
comparisons used by the sorting algorithms in Question 3 
when used to sort sequences that only have a small fraction 
of terms out of order. 


Write a program that animates the progress of all the sort- 
ing algorithms in Question 3 when given the numbers from 
1 to 100 in random order. 


. Develop a detailed list of algorithmic paradigms and pro- 


vide examples using each of these paradigms. 


. Explain what the Turing Award is and describe the cri- 


teria used to select winners. List six past winners of the 
award and why they received the award. 


Describe what is meant by a parallel algorithm. Explain 
how the pseudocode used in this book can be extended to 
handle parallel algorithms. 


Explain how the complexity of parallel algorithms can be 
measured. Give some examples to illustrate this concept, 
showing how a parallel algorithm can work more quickly 
than one that does not operate in parallel. 


Describe six different NP-complete problems. 


Demonstrate how one of the many different NP-complete 
problems can be reduced to the satisfiability problem. 


4.1 Divisibility and 
Modular 
Arithmetic 


4.2 Integer Repre- 
sentations and 
Algorithms 


4.3 Primes and 
Greatest 
Common 
Divisors 


4.4 Solving 
Congruences 


4.5 Applications of 
Congruences 


4.6 Cryptography 


Number Theory and Cryptography 


T he part of mathematics devoted to the study of the set of integers and their properties is 
known as number theory. In this chapter we will develop some of the important concepts 
of number theory including many of those used in computer science. As we develop number 
theory, we will use the proof methods developed in Chapter 1 to prove many theorems. 

We will first introduce the notion of divisibility of integers, which we use to introduce 
modular, or clock, arithmetic. Modular arithmetic operates with the remainders of integers when 
they are divided by a fixed positive integer, called the modulus. We will prove many important 
results about modular arithmetic which we will use extensively in this chapter. 

Integers can be represented with any positive integer b greater than 1 as a base. In this 
chapter we discuss base b representations of integers and give an algorithm for finding them. In 
particular, we will discuss binary, octal, and hexadecimal (base 2, 8, and 16) representations. 
We will describe algorithms for carrying out arithmetic using these representations and study 
their complexity. These algorithms were the first procedures called algorithms. 

We will discuss prime numbers, the positive integers that have only 1 and themselves as 
positive divisors. We will prove that there are infinitely many primes; the proof we give is con- 
sidered to be one of the most beautiful proofs in mathematics. We will discuss the distribution 
of primes and many famous open questions concerning primes. We will introduce the concept 
of greatest common divisors and study the Euclidean algorithm for computing them. This algo- 
rithm was first described thousands of years ago. We will introduce the fundamental theorem 
of arithmetic, a key result which tells us that every positive integer has a unique factorization 
into primes. 

We will explain how to solve linear congruences, as well as systems of linear congruences, 
which we solve using the famous Chinese remainder theorem. We will introduce the notion of 
pseudoprimes, which are composite integers masquerading as primes, and show how this notion 
can help us rapidly generate prime numbers. 

This chapter introduces several important applications of number theory. In particular, we 
will use number theory to generate pseudorandom numbers, to assign memory locations to 
computer files, and to find check digits used to detect errors in various kinds of identification 
numbers. We also introduce the subject of cryptography. Number theory plays an essential role 
both in classical cryptography, first used thousands of years ago, and modern cryptography, 
which plays an essential role in electronic communication. We will show how the ideas we 
develop can be used in cryptographical protocols, introducing protocols for sharing keys and for 
sending signed messages. Number theory, once considered the purest of subjects, has become 
an essential tool in providing computer and Internet security. 

Finally, it should be noted that this chapter is designed to introduce some key aspects of 
number theory. As with all the topics covered in this book, there is a great deal more to learn. 
Interested students can consult [Ro11], the author’s number theory text, to explore this fasci- 
nating subject more fully. 


Divisibility and Modular Arithmetic 


4.1.1 Introduction 


The ideas that we will develop in this section are based on the notion of divisibility. Division 
of an integer by a positive integer produces a quotient and a remainder. Working with these re- 
mainders leads to modular arithmetic, which plays an important role in mathematics and which 
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Definition 1 


EXAMPLE 1 


EXAMPLE 2 


Extra » 
Examples 


THEOREM 1 


is used throughout computer science. We will discuss some important applications of modular 
arithmetic later in this chapter, including generating pseudorandom numbers, assigning com- 
puter memory locations to files, constructing check digits, and encrypting messages. 


4.1.2 Division 


When one integer is divided by a second nonzero integer, the quotient may or may not be 
an integer. For example, 12/3 = 4 is an integer, whereas 11/4 = 2.75 is not. This leads to 
Definition 1. 


If a and b are integers with a # 0, we say that a divides b if there is an integer c such that 
b = ac (or equivalently, if 2 is an integer). When a divides b we say that a is a factor or 
divisor of b, and that b is a multiple of a. The notation a | b denotes that a divides b. We 
write a f b when a does not divide b. 


Remark: We can express a | b using quantifiers as 3c(ac = b), where the universe of discourse 
is the set of integers. 


In Figure 1 a number line indicates which integers are divisible by the positive integer d. 
Determine whether 3 | 7 and whether 3 | 12. 
Solution: We see that 3 ¥ 7, because 7/3 is not an integer. On the other hand, 3 | 12 because 
12/3 = 4. < 
Let n and d be positive integers. How many positive integers not exceeding n are divisible by d? 
Solution: The positive integers divisible by d are all the integers of the form dk, where k is 
a positive integer. Hence, the number of positive integers divisible by d that do not exceed n 


equals the number of integers k with 0 < dk <n, or with O < k < n/d. Therefore, there are 
[n/d] positive integers not exceeding n that are divisible by d. < 


Some of the basic properties of divisibility of integers are given in Theorem 1. 


Let a, b, and c be integers, where a # 0. Then 


(i) ifa | b anda |c, then a | (b+ c); 
(ii) if a | b, then a | bc for all integers c; 
(iii) ifa|bandb|c, thena|c. 


—3d =2d -=d 0 d 2d 3d 


FIGURE 1 Integers divisible by the positive integer d. 


COROLLARY 1 


THEOREM 2 


Definition 2 


EXAMPLE 3 
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Proof: We will give a direct proof of (i). Suppose that a | b and a | c. Then, from the definition 
of divisibility, it follows that there are integers s and t with b = as and c = at. Hence, 
b+c=astat=a(st+t). 


Therefore, a divides b + c. This establishes part (7) of the theorem. The proofs of parts (ii) and 
(iii) are left as Exercises 3 and 4. < 


Theorem 1 has this useful consequence. 


If a, b, and c are integers, where a # 0, such that a | b and a | c, then a | mb + nc whenever 
m and n are integers. 


Proof: We will give a direct proof. By part (ii) of Theorem 1 we see that a | mb and a | nc 
whenever m and n are integers. By part (i) of Theorem 1 it follows that a | mb + nc. 


4.1.3 The Division Algorithm 


When an integer is divided by a positive integer, there is a quotient and a remainder, as the 
division algorithm shows. 


THE DIVISION ALGORITHM Leta be an integer and d a positive integer. Then there 
are unique integers q and r, with O < r < d, such that a = dq + r. 


We defer the proof of the division algorithm to Section 5.2. (See Example 5 and 
Exercise 37 in that section.) 


Remark: Theorem 2 is not really an algorithm. (Why not?) Nevertheless, we use its traditional 
name. 


In the equality given in the division algorithm, d is called the divisor, a is called the dividend, 
q is called the quotient, and r is called the remainder. This notation is used to express the 
quotient and remainder: 


q=adivd, r=amodd. 
Remark: Note that both a div d and a mod d for a fixed d are functions on the set of inte- 


gers. Furthermore, when a is an integer and d is a positive integer, we have a div d = |a/d| 
and a mod d = a — d. (See Exercise 24.) 


Examples 3 and 4 illustrate the division algorithm. 
What are the quotient and remainder when 101 is divided by 11? 
Solution: We have 

101=11-9+2. 


Hence, the quotient when 101 is divided by 11 is 9= 101 div 11, and the remainder is 
2= 101 mod 11. < 
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EXAMPLE 4 


Extra > 
Examples 


Definition 3 


THEOREM 3 


What are the quotient and remainder when —11 is divided by 3? 
Solution: We have 
-11 =3(-4) +1. 


Hence, the quotient when —11 is divided by 3 is —4 = —11 div 3, and the remainder is 
1 = —11 mod 3. 

Note that the remainder cannot be negative. Consequently, the remainder is not —2, even 
though 


-11 = 3(-3) - 2, 


because r = —2 does not satisfy 0 < r < 3. < 


Note that the integer a is divisible by the integer d if and only if the remainder is zero 
when a is divided by d. 


Remark: A programming language may have one, or possibly two, operators for modular arith- 
metic, denoted by mod (in BASIC, Maple, Mathematica, EXCEL, and SQL), % (in C, C++, 
Java, and Python), rem (in Ada and Lisp), or something else. Be careful when using them, be- 
cause for a < 0, some of these operators return a — m[a/m] instead of a mod m = a — m|a/m] 
(as shown in Exercise 24). Also, unlike a mod m, some of these operators are defined when 
m < 0, and even when m = 0. 


4.1.4 Modular Arithmetic 


In some situations we care only about the remainder of an integer when it is divided by some 
specified positive integer. For instance, when we ask what time it will be (on a 24-hour clock) 
50 hours from now, we care only about the remainder when 50 plus the current hour is divided 
by 24. Because we are often interested only in remainders, we have special notations for them. 
We have already introduced the notation a mod m to represent the remainder when an inte- 
ger a is divided by the positive integer m. We now introduce a different, but related, notation 
that indicates that two integers have the same remainder when they are divided by the positive 
integer m. 


If a and b are integers and m is a positive integer, then a is congruent to b modulo m if m di- 
vides a — b. We use the notation a = b (mod m) to indicate that a is congruent to b modulo m. 
We say that a = b (mod m) is a congruence and that m is its modulus (plural moduli). If a 
and b are not congruent modulo m, we write a  b (mod m). 


Although both notations a = b (mod m) and a mod m = b include “mod,” they represent fun- 
damentally different concepts. The first represents a relation on the set of integers, whereas the 
second represents a function. However, the relation a = b (mod m) and the mod m function are 
closely related, as described in Theorem 3. 


Let a and b be integers, and let m be a positive integer. Then a = b (mod m) if and only 
if amod m = b mod m. 


EXAMPLE 5 


THEOREM 4 


THEOREM 5 
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The proof of Theorem 3 is left as Exercises 21 and 22. Recall that a mod m and b mod m are 
the remainders when a and b are divided by m, respectively. Consequently, Theorem 3 also says 
that a = b (mod m) if and only if a and b have the same remainder when divided by m. 


Determine whether 17 is congruent to 5 modulo 6 and whether 24 and 14 are congruent 
modulo 6. 


Solution: Because 6 divides 17 — 5 = 12, we see that 17 =5 (mod 6). However, because 
24 — 14 = 10 is not divisible by 6, we see that 24 # 14 (mod 6). < 


The great German mathematician Karl Friedrich Gauss developed the concept of congru- 
ences at the end of the eighteenth century. The notion of congruences has played an important 
role in the development of number theory. 

Theorem 4 provides a useful way to work with congruences. 


Let m be a positive integer. The integers a and b are congruent modulo m if and only if there 
is an integer k such that a = b + km. 


Proof: If a=b (mod m), by the definition of congruence (Definition 3), we know that 
m | (a — b). This means that there is an integer k such that a — b = km, so that a = b + km. 
Conversely, if there is an integer k such that a = b + km, then km = a — b. Hence, m divides 
a — b, so that a = b (mod m). < 


The set of all integers congruent to an integer a modulo m is called the congruence class 
of a modulo m. In Chapter 9 we will show that there are m pairwise disjoint equivalence classes 
modulo m and that the union of these equivalence classes is the set of integers. 

Theorem 5 shows that additions and multiplications preserve congruences. 


Let m be a positive integer. If a = b (mod m) and c = d (mod m), then 


a+c=b+d (mod m) and ac = bd (mod m). 


KARL FRIEDRICH GAUSS (1777-1855) Karl Friedrich Gauss, the son of a bricklayer, was a child prodigy. 
He demonstrated his potential at the age of 10, when he quickly solved a problem assigned by a teacher to 
keep the class busy. The teacher asked the students to find the sum of the first 100 positive integers. Gauss 
realized that this sum could be found by forming 50 pairs, each with the sum 101: 1 + 100, 2 + 99, ..., 50 + 51. 
This brilliance attracted the sponsorship of patrons, including Duke Ferdinand of Brunswick, who made it 
possible for Gauss to attend Caroline College and the University of Göttingen. While a student, he invented 
the method of least squares, which is used to estimate the most likely value of a variable from experimental 
results. In 1796 Gauss made a fundamental discovery in geometry, advancing a subject that had not advanced 
since ancient times. He showed that a 17-sided regular polygon could be drawn using just a ruler and compass. 

In 1799 Gauss presented the first rigorous proof of the fundamental theorem of algebra, which states 
that a polynomial of degree n has exactly n roots in the complex numbers (counting multiplicities). 


Gauss achieved worldwide fame when he successfully calculated the orbit of the first asteroid discovered, Ceres, using scanty 


data. 


Gauss was called the Prince of Mathematics by his contemporary mathematicians. Although Gauss is noted for his many 
discoveries in geometry, algebra, analysis, astronomy, and physics, he had a special interest in number theory, which can be seen 
from his statement “Mathematics is the queen of the sciences, and the theory of numbers is the queen of mathematics.” Gauss laid 
the foundations for modern number theory with the publication of his book Disquisitiones Arithmeticae in 1801. 
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EXAMPLE 6 


? 


You cannot always 
divide both sides 

of a congruence 

by the same number! 


COROLLARY 2 


Proof: We use a direct proof. Because a = b (mod m) and c = d (mod m), by Theorem 4 there 
are integers s and ¢ with b = a + sm and d = c + tm. Hence, 


b +d = (a+ sm)+ (c +tm)=(a+c)+ ms +t) 
and 

bd = (a + sm)(c + tm) = ac + m(at + cs + stm). 
Hence, 


a+c =b +d (mod m) and ac = bd (mod m). < 


Because 7 = 2 (mod 5) and 11 = 1 (mod 5), it follows from Theorem 5 that 

18=74+ 11 =2+4+1=3 (mod 5) 
and that 

77=7-11=2-1=2 (mod 5). < 

We must be careful working with congruences. Some properties we may expect to be true 
are not valid. For example, if ac = bc (mod m), the congruence a = b (mod m) may be false. 
Similarly, if a = b (mod m) and c = d (mod m), the congruence a° = b? (mod m) may be false. 
(See Exercise 43.) 

Corollary 2 shows how to find the values of the mod m function at the sum and product of 


two integers using the values of this function at each of these integers. We will use this result 
in Section 5.4. 


Let m be a positive integer and let a and b be integers. Then 
(a + b) mod m = ((a mod m) + (b mod m)) mod m 
and 


ab mod m = ((a mod m)(b mod m)) mod m. 


Proof: By the definitions of modm and of congruence modulo m, we know that a = 
(a mod m) (mod m) and b = (b mod m) (mod m). Hence, Theorem 5 tells us that 


a + b = (a mod m) + (b mod m) (mod m) 
and 
ab = (a mod m)(b mod m) (mod m). 


The equalities in this corollary follow from these last two congruences by Theorem 3. < 


EXAMPLE 7 


EXAMPLE 8 
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In Section 4.6 we will carry out a variety of computations using the mod function when 
we study cryptography. Example 7 illustrates a type of computation involving the mod function 
that we will encounter. 


Find the value of (19° mod 31)* mod 23. 


Solution: To compute (19? mod31)* mod 23, we will first evaluate 19° mod31. Be- 
cause 193 = 6859 and 6859 = 221 -31 +8, we have 19° mod31 = 6859 mod 31 = 8. So, 
(19° mod 31)* mod 23 = 84 mod 23. 

Next, note that 84 = 4096. Because 4096 = 178 - 23 + 2, we have 4096 mod 23 = 2. Hence, 
(193 mod 31)* mod 23 = 2. < 


4.1.5 Arithmetic Modulo m 


We can define arithmetic operations on Z, the set of nonnegative integers less than m, that is, 
the set {0, 1, ..., m — 1}. In particular, we define addition of these integers, denoted by +,„ by 


a+t,, b = (a+ b) mod m, 


where the addition on the right-hand side of this equation is the ordinary addition of integers, 
and we define multiplication of these integers, denoted by -,,, by 


a:m b =(a-b)modm, 


where the multiplication on the right-hand side of this equation is the ordinary multiplication of 
integers. The operations +,, and -,, are called addition and multiplication modulo m and when 
we use these operations, we are said to be doing arithmetic modulo m. 


Use the definition of addition and multiplication in Z, to find 7 +4; 9 and 7 -,, 9. 
Solution: Using the definition of addition modulo 11, we find that 

7+; 9=(7 +9) mod 11 = 16 mod 11 =5, 
and 

7-19 = (7 -9)mod 11 = 63 mod 11 = 8. 


Hence, 7+,,9=5 and7 -9 = 8. < 


The operations +,, and -,, satisfy many of the same properties of ordinary addition and 
multiplication of integers. In particular, they satisfy these properties: 


Closure If a and b belong to Z,,,, then a+,, b and a -,, b belong to Z,- 


m? 


(b+,,c) and 


m m 


Associativity If a, b, and c belong to Z 
(nD) iy C= Ay (Din C)- 


then (a+,,b)+,,¢ =at 


m? 


m m 


Commutativity If a and b belong to Z,,, thena+,,b=b+,,aanda-,,b=b.-,, a. 


m? m 


Identity elements The elements 0 and 1 are identity elements for addition and multiplication 
modulo m, respectively. That is, if a belongs to Z,,, thena+,,0=0+,,a=aanda-,1= 
1-,a@=4. 


m? m 


then m — a is an additive inverse of a modulo m and 
(m—a)=Oand0+,,0=0. 


Additive inverses If a 4 0 belongs to Z,,, 


O is its own additive inverse. That is, a +, 


258 4/ Number Theory and Cryptography 


Distributivity If a, b, and c belong to Z 
(a En b) “m c = (a “m c) F (b “m c). 


m? then a ‘m (b Fin c) = (a `m b) Fin (a ‘m c) and 


These properties follow from the properties we have developed for congruences and remainders 
modulo m, together with the properties of integers; we leave their proofs as Exercises 48—50. 
Note that we have listed the property that every element of Z,, has an additive inverse, but no 
analogous property for multiplicative inverses has been included. This is because multiplicative 
inverses do not always exist modulo m. For instance, there is no multiplicative inverse of 2 
modulo 6, as the reader can verify. We will return to the question of when an integer has a 
multiplicative inverse modulo m later in this chapter. 


Remark: Because Z, with the operations of addition and multiplication modulo m satisfies the 
properties listed, Z,,, with modular addition is said to be a commutative group and Z, with both 
of these operations is said to be a commutative ring. Note that the set of integers with ordinary 
addition and multiplication also forms a commutative ring. Groups and rings are studied in 
courses that cover abstract algebra. 


Remark: In Exercise 36, and in later sections, we will use the notations + and - for +, and -,, 
without the subscript m on the symbol for the operator whenever we work with Z. 


Exercises 
1. Does 17 divide each of these numbers? d) —1 is divided by 23? 
a) 68 c) 357 d) 1001 e) —2002 is divided by 87? 
2. Prove that if a is an integer other than 0, then f) Ois divided by 17? 
a) ldividesa. b) a divides 0. g) 1,234,567 is divided by 1001? 
3. Prove that part (ii ) of Theorem 1 is true. h) —100 is divided by 101? 


4. Prove that part (iii ) of Theorem 1 is true. 


15. What time does a 12-hour clock read 


5. Show that if a | b and b | a, where a and b are integers, a) 80 hours after it reads 11:00? 
then a = b or a = —b. b) 40 hours before it reads 12:00? 
6. Show that if a, b, c, and d are integers, where a # 0, such c) 100 hours after it reads 6:00? 
that a | c and b | d, then ab | cd. 16. What time does a 24-hour clock read 
7. Show that if a, b, and c are integers, where a # 0 and a) 100 hours after it reads 2:00? 
c #0, such that ac | bc, then a | b. b) 45 hours before it reads 12:00? 
8. Prove or disprove that if a | bc, where a, b, and c are pos- c) 168 hours after it reads 19:00? 
itive integers and a # 0, then a | b or a | c. 17. Suppose that a and b are integers, a = 4(mod 13), and 
9. Prove that if a and b are integers and a divides b, then a b = 9(mod 13). Find the integer c with 0 < c < 12 such 
is odd or b is even. that 
10. Prove that if a and b are nonzero integers, a divides b, a) c = 9a(mod 13). 
and a + b is odd, then a is odd. b) c = 11b(mod 13). 
11. Prove that if a is an integer that is not divisible by 3, then c) c = a+b(mod 13). 
(a + 1)(a + 2) is divisible by 3. d) c = 2a + 3b (mod 13). 
12. Prove that if a is a positive integer, then 4 does not divide e) c = a’ +b’ (mod 13). 
a +2. f) c = a — b (mod 13). 
13. What are the quotient and remainder when 18. Suppose that a and b are integers, a = 11 (mod 19), and 
a) 19 is divided by 7? b) —111 is divided by 11? b = 3(mod 19). Find the integer c with O < c < 18 such 
c) 789 is divided by 23? d) 1001 is divided by 13? that 
e) Ois divided by 19? f) 3 is divided by 5? a) c = 13a(mod 19). 
g) —1 is divided by 3? h) 4 is divided by 1? b) c = 8b(mod 19). 
14. What are the quotient and remainder when c) c = a — b (mod 19). 
a) 44 is divided by 8? d) c = 7a + 3b (mod 19). 
b) 777 is divided by 21? e) c = 2a’ + 3b’ (mod 19). 
c) —123 is divided by 19? f) c = @ + 4b3 (mod 19). 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 
33. 


34. 


35. 


36. 


Show that if a and d are positive integers, then 
(—a) div d = —a div d if and only if d divides a. 

Prove or disprove that if a, b, and d are integers with 
d > 0, then (a + b) div d = a div d + b div d. 

Let m be a positive integer. Show that a = b (mod m) if a 
mod m = b mod m. 

Let m be a positive integer. Show that a mod m = 
b mod m if a = b (mod m). 

Show that if n and k are positive integers, then [n/k] 
[(n—1)/k] +1. 

Show that if a is an integer and d is an integer 
greater than 1, then the quotient and remainder obtained 
when a is divided by d are |a/d| and a-— dļ|a/d], 
respectively. 

Find a formula for the integer with smallest absolute 
value that is congruent to an integer a modulo m, where 
m is a positive integer. 
Evaluate these quantities. 
a) —17 mod 2 

c) —101 mod 13 


Evaluate these quantities. 


a) 13 mod 3 b) —97 mod 11 
c) 155 mod 19 d) —221 mod 23 
Find a div m and a mod m when 

a) a = —111, m = 99. 

b) a = —9999, m = 101. 

c) a= 10299, m = 999. 

d) a = 123456, m = 1001. 

Find a div m and a mod m when 

a) a = 228, m = 119. 

b) a = 9009, m = 223. 

c) a = —10101, m = 333. 

d) a = —765432, m = 38271. 

Find the integer a such that 

a) a = 43 (mod 23) and —22 <a < 0. 

b) a = 17 (mod 29) and -14 <a < 14. 

c) a = —11(mod21) and 90 <a < 110. 
Find the integer a such that 

a) a = —15 (mod 27) and —26 < a < 0. 

b) a = 24(mod31) and -15 <a < 15. 

c) a = 99 (mod 41) and 100 < a < 140. 

List five integers that are congruent to 4 modulo 12. 
List all integers between —100 and 100 that are congruent 
to —1 modulo 25. 


b) 144 mod 7 
d) 199 mod 19 


Decide whether each of these integers is congruent to 
3 modulo 7. 

a) 37 b) 66 

c) -17 d) —67 

Decide whether each of these integers is congruent to 
5 modulo 17. 

a) 80 b) 103 

c) —29 d) —122 

Find each of these values. 


a) (177 mod 31 + 270 mod 31) mod 31 
b) (177 mod 31 - 270 mod 31) mod 31 


37. 


38. 


39. 


40. 


41. 


43. 


44. 
45. 


46. 


47. 


48. 


49. 


50. 


51. 


52. 


53. 
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Find each of these values. 


a) (—133 mod 23 + 261 mod 23) mod 23 

b) (457 mod 23 - 182 mod 23) mod 23 

Find each of these values. 

a) (192 mod 41) mod 9 

b) (32° mod 13)? mod 11 

c) (7? mod 23)? mod 31 

d) (21? mod 15) mod 22 

Find each of these values. 

a) (997 mod 32)? mod 15 

b) (34mod 17)? mod 11 

c) (19° mod 23)? mod 31 

d) (89° mod 79)* mod 26 

Show that if a = b (mod m) and c = d (mod m), where 
a, b, c, d, and m are integers with m > 2, then a — c = 
b — d (mod m). 

Show that if n | m, where n and m are integers greater 
than 1, and if a = b (mod m), where a and b are integers, 
then a = b (mod n). 


. Show that if a, b, c, and m are integers such that m > 2, 


c > 0, and a = b (mod m), then ac = bc (mod mc). 

Find counterexamples to each of these statements about 

congruences. 

a) If ac = bc (mod m), where a, b, c, and m are integers 
with m > 2, then a = b (mod m). 

b) If a=b (mod m) and c=d (mod m), where 
a, b, c, d, and m are integers with c and d positive 
and m > 2, then a° = bf (mod m). 

Show that if n is an integer then n? = 0 or 1 (mod 4). 

Use Exercise 44 to show that if m is a positive integer of 

the form 4k + 3 for some nonnegative integer k, then m 

is not the sum of the squares of two integers. 

Prove that if n is an odd positive integer, then n? = 

1 (mod 8). 

Show that if a, b, k, and m are integers such that k > 1, 

m > 2, and a = b (mod m), then a = b‘(mod m). 

Show that Z,, with addition modulo m, where m > 2 is 

an integer, satisfies the closure, associative, and commu- 

tative properties, 0 is an additive identity, and for every 

nonzero a € Z,,, m — ais an inverse of a modulo m. 

Show that Z, with multiplication modulo m, where m > 

2 is an integer, satisfies the closure, associative, and com- 

mutativity properties, and 1 is a multiplicative identity. 

Show that the distributive property of multiplication over 

addition holds for Z,,, where m > 2 is an integer. 

Write out the addition and multiplication tables for 

Z; (where by addition and multiplication we mean +5 

and -;). 

Write out the addition and multiplication tables for 

Zs (where by addition and multiplication we mean +6 

and -¢). 

Determine whether each of the functions f(a) = a div d 

and g(a) = a mod d, where d is a fixed positive integer, 

from the set of integers to the set of integers, is one-to- 
one, and determine whether each of these functions is 
onto. 


m? 
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Integer Representations and Algorithms 


THEOREM 1 


4.2.1 Introduction 


Integers can be expressed using any integer greater than one as a base, as we will show in this 
section. Although we commonly use decimal (base 10), representations, binary (base 2), octal 
(base 8), and hexadecimal (base 16) representations are often used, especially in computer sci- 
ence. Given a base b and an integer n, we will show how to construct the base b representation of 
this integer. We will also explain how to quickly convert between binary and octal and between 
binary and hexadecimal notations. 

As mentioned in Section 3.1, the term algorithm originally referred to procedures for per- 
forming arithmetic operations using the decimal representations of integers. These algorithms, 
adapted for use with binary representations, are the basis for computer arithmetic. They provide 
good illustrations of the concept of an algorithm and the complexity of algorithms. For these 
reasons, they will be discussed in this section. 

We will also introduce an algorithm for finding a div d and a mod d where a and d are 
integers with d > 1. Finally, we will describe an efficient algorithm for modular exponentiation, 
which is a particularly important algorithm for cryptography, as we will see in Section 4.6. 


4.2.2 Representations of Integers 


In everyday life we use decimal notation to express integers. In decimal notation, an integer n 
is written as a sum of the form a, 10* + a,_, 104! + --- + a, 10 + ap, where a; is an integer with 
0< a; S 9 for j = 0, 1, ..., k. For example, 965 is used to denote 9 - 10? + 6 - 10 + 5. However, 
it is often convenient to use bases other than 10. In particular, computers usually use binary 
notation (with 2 as the base) when carrying out arithmetic, and octal (base 8) or hexadecimal 
(base 16) notation when expressing characters, such as letters or digits. In fact, we can use any 
integer greater than 1 as the base when expressing integers. This is stated in Theorem 1. 


Let b be an integer greater than 1. Then if n is a positive integer, it can be expressed uniquely 
in the form 


n= a,b tapib + tab + a, 


where k is a nonnegative integer, do, a), ..., a, are nonnegative integers less than b, and 


a, # 0. 


A proof of this theorem can be constructed using mathematical induction, a proof method that 
is discussed in Section 5.1. It can also be found in [Ro10]. The representation of n given in 
Theorem | is called the base b expansion of n. The base b expansion of n is denoted by 
(aa; --- 4,49)». For instance, (245), represents 2 - 8? + 4-8 +5 = 165. Typically, the sub- 
script 10 is omitted for base 10 expansions of integers because base 10, or decimal expansions, 
are commonly used to represent integers. 


BINARY EXPANSIONS Choosing 2 as the base gives binary expansions of integers. In bi- 
nary notation each digit is either a 0 or a 1. In other words, the binary expansion of an integer 
is just a bit string. Binary expansions (and related expansions that are variants of binary expan- 
sions) are used by computers to represent and do arithmetic with integers. 


EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 
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What is the decimal expansion of the integer that has (1 0101 1111), as its binary expansion? 


Solution: We have 
(10101 1111) =1-28+ 05-2) 4142? 062? 42? 
$1929 41.9741 2! 41229 = 351, < 


OCTAL AND HEXADECIMAL EXPANSIONS Among the most important bases in com- 
puter science are base 2, base 8, and base 16. Base 8 expansions are called octal expansions and 
base 16 expansions are hexadecimal expansions. 
What is the decimal expansion of the number with octal expansion (7016).? 
Solution: Using the definition of a base b expansion with b = 8 tells us that 
(7016), =7-8°+0-87+1-8+6= 3598. < 
Sixteen different digits are required for hexadecimal expansions. Usually, the hexadecimal 


digits used are 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, and F, where the letters A through F 
represent the digits corresponding to the numbers 10 through 15 (in decimal notation). 


What is the decimal expansion of the number with hexadecimal expansion (2AE0B) 6? 
Solution: Using the definition of a base b expansion with b = 16 tells us that 
(2AE0B),, = 2-16*+ 10-16°+ 14-167 + 0-16+11 = 175627. < 
Each hexadecimal digit can be represented using four bits. For instance, we see that 


(1110 0101), = (E5),, because (1110), = (E),, and (0101), = (5),¢. Bytes, which are bit 
strings of length eight, can be represented by two hexadecimal digits. 


BASE CONVERSION We will now describe an algorithm for constructing the base b expan- 
sion of an integer n. First, divide n by b to obtain a quotient and remainder, that is, 


n = bqo + ap, O<a <b. 


The remainder, dp, is the rightmost digit in the base b expansion of n. Next, divide qọ by b to 
obtain 


qo = bq, +4}, O<a, <b. 
We see that a, is the second digit from the right in the base b expansion of n. Continue this 
process, successively dividing the quotients by b, obtaining additional base b digits as the re- 
mainders. This process terminates when we obtain a quotient equal to zero. It produces the base 


b digits of n from the right to the left. 


Find the octal expansion of (12345), 9. 


Extra » Solution: First, divide 12345 by 8 to obtain 


Examples 


12345 = 8 - 1543 +1. 
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EXAMPLE 5 


EXAMPLE 6 


Successively dividing quotients by 8 gives 


1543 = 8-192 +7, 


192 = 8 -24 + 0, 
24=8.3+0, 
3=8.0+3. 


The successive remainders that we have found, 1, 7, 0, 0, and 3, are the digits from the right to 


the left of 12345 in base 8. Hence, 
(12345) ) = (30071). i 


Find the hexadecimal expansion of (177130),o. 


Solution: First divide 177130 by 16 to obtain 
177130 = 16- 11070 + 10. 
Successively dividing quotients by 16 gives 


11070 = 16-691 + 14, 


691 = 16-43 +3, 
43 =16-2+11, 
2=16-0+2. 


The successive remainders that we have found, 10, 14, 3, 11, 2, give us the digits from the right 
to the left of 177130 in the hexadecimal (base 16) expansion of (177130),9. It follows that 


(177130) 1) = (2B3EA),¢. 


(Recall that the integers 10, 11, and 14 correspond to the hexadecimal digits A, B, and E, re- 
spectively.) < 


Find the binary expansion of (241). 


Solution: First divide 241 by 2 to obtain 
241 =2-120+1. 


Successively dividing quotients by 2 gives 


120 =2-60+0, 
60 = 2-30+0, 
30=2-15+4+0, 
15=2-7+1, 
7=2-3+441, 
3=2-1+1, 
1=2-0+1. 


The successive remainders that we have found, 1, 0, 0, 0, 1, 1, 1, 1, are the digits from the right 
to the left in the binary (base 2) expansion of (241) 9. Hence, 


(241),) = (1111 0001). a 


The pseudocode given in Algorithm 1 finds the base b expansion (a;_) ... 4,9), of the 
integer n. 
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TABLE 1 Hexadecimal, Octal, and Binary Representation of the Integers 0 through 15. 


ALGORITHM 1 Constructing Base b Expansions. 


procedure base b expansion(n, b: positive integers with b > 1) 


qian 
k:=0 
while q 4 0 
a, := q mod b 
q := q diy b 
k:=k+1 
return (a;_p ..., A1 4o) {(ay_1 --- 414o), is the base b expansion of n} 


In Algorithm 1, q represents the quotient obtained by successive divisions by b, starting with 
q = n. The digits in the base b expansion are the remainders of these divisions and are given by 
q mod b. The algorithm terminates when a quotient q = 0 is reached. 


Remark: Note that Algorithm 1 can be thought of as a greedy algorithm, because the base b 
digits are taken as large as possible in each step. 


CONVERSION BETWEEN BINARY, OCTAL, AND HEXADECIMAL EXPANSIONS 
Conversion between binary and octal and between binary and hexadecimal expansions is ex- 
tremely easy because each octal digit corresponds to a block of three binary digits and each hex- 
adecimal digit corresponds to a block of four binary digits, with these correspondences shown 
in Table 1 without initial Os shown. (We leave it as Exercises 13—16 to show that this is the 
case.) This conversion is illustrated in Example 7. 


EXAMPLE 7 Find the octal and hexadecimal expansions of (11 1110 1011 1100), and the binary expansions 
of (765), and (A8D),¢. 


Solution: To convert (11 1110 1011 1100), into octal notation we group the binary dig- 
its into blocks of three, adding initial zeros at the start of the leftmost block if necessary. 
These blocks, from left to right, are 011, 111, 010, 111, and 100, corresponding to 3, 7, 2, 7, 
and 4, respectively. Consequently, (11 1110 1011 1100), = (37274). To convert (11 1110 1011 
1100), into hexadecimal notation we group the binary digits into blocks of four, adding initial 
zeros at the start of the leftmost block if necessary. These blocks, from left to right, are 0011, 
1110, 1011, and 1100, corresponding to the hexadecimal digits 3, E, B, and C, respectively. 
Consequently, (11 1110 1011 1100), = (3EBC),¢. 

To convert (765)¢ into binary notation, we replace each octal digit by a block of three binary 
digits. These blocks are 111, 110, and 101. Hence, (765), = (1 1111 0101),. To convert (A8D);6 
into binary notation, we replace each hexadecimal digit by a block of four binary digits. These 
blocks are 1010, 1000, and 1101. Hence, (A8D); = (1010 1000 1101),. 
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EXAMPLE 8 


4.2.3 Algorithms for Integer Operations 


The algorithms for performing operations with integers using their binary expansions are ex- 
tremely important in computer arithmetic. We will describe algorithms for the addition and the 
multiplication of two integers expressed in binary notation. We will also analyze the compu- 
tational complexity of these algorithms, in terms of the actual number of bit operations used. 
Throughout this discussion, suppose that the binary expansions of a and b are 


a = (Ay_14y_7 ++» 4, Ag)9 b = (b,_1,_9 --- b1b0)2; 
so that a and b each have n bits (putting bits equal to 0 at the beginning of one of these expansions 
if necessary). 

We will measure the complexity of algorithms for integer arithmetic in terms of the number 
of bits in these numbers. 
ADDITION ALGORITHM Consider the problem of adding two integers in binary notation. 
A procedure to perform addition can be based on the usual method for adding numbers with 
pencil and paper. This method proceeds by adding pairs of binary digits together with carries, 
when they occur, to compute the sum of two integers. This procedure will now be specified 


in detail. 
To add a and b, first add their rightmost bits. This gives 


do + bo = C: 2 + Sos 


where sọ is the rightmost bit in the binary expansion of a + b and cg is the carry, which is either 
O or 1. Then add the next pair of bits and the carry, 


a, +b, +c) =c,-2+5), 


where s, is the next bit (from the right) in the binary expansion of a + b, and c} is the carry. 
Continue this process, adding the corresponding bits in the two binary expansions and the carry, 
to determine the next bit from the right in the binary expansion of a + b. At the last stage, 
add a,_;, b,_,, and c,_» to obtain c,_,; -2+,_,. The leading bit of the sum is s, = c,_,. This 
procedure produces the binary expansion of the sum, namely, a + b = (5,5,_15,_9 «+. 51Sq)o- 


Add a = (1110), and b = (1011)5. 


Solution: Following the procedure specified in the algorithm, first note that 
dg tbo =04+1=0-241, 

so that cy = 0 and sy = 1. Then, because 
aq, +b,;+c9=14+1+0=1-2+0, 

it follows that c} = 1 ands, = 0. Continuing, 


ad, +by+c,=14+0+4+1=1-2+0, 


IJI 
1110 
+1011 


11001 
FIGURE 1 


Adding (1110), 
and (1011),. 


EXAMPLE 9 
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so that c, = 1 and s, = 0. Finally, because 


az +b, +c =1+1+1=1.2+1, 


follows that c, = 1 and s; = 1. This means that sy = c; = 1. Therefore, s = a + b = (1 1001). 
This addition is displayed in Figure 1, where carries are shown in color. < 


The algorithm for addition can be described using pseudocode as follows. 


ALGORITHM 2 Addition of Integers. 


procedure add(a, b: positive integers) 
{the binary expansions of a and b are (a„_14,—2 --- 4,d)o 
and (b„—1b,-2 .-. b1bo)2, respectively } 
c:=0 
for j:=Oton-1 
d := | (a; + b; + c)/2] 
sj = a,+b, +c- 2d 


n-1 


c:=d 
se 
return (so, S1, ..., Sp) {the binary expansion of the sum is (S„S„—1 --- So)2} 


Next, the number of additions of bits used by Algorithm 2 will be analyzed. 


How many additions of bits are required to use Algorithm 2 to add two integers with n bits (or 
less) in their binary representations? 


Solution: Two integers are added by successively adding pairs of bits and, when it occurs, a 
carry. Adding each pair of bits and the carry requires two additions of bits. Thus, the total 
number of additions of bits used is less than twice the number of bits in the expansion. Hence, 
the number of additions of bits used by Algorithm 2 to add two n-bit integers is O(n). < 


MULTIPLICATION ALGORITHM Next, consider the multiplication of two n-bit integers a 
and b. The conventional algorithm (used when multiplying with pencil and paper) works as 
follows. Using the distributive law, we see that 


ab= a(by2° + b,2! he. Sais ahs b, 12") 
= a(by2°) + a(b, 2!) + +++ + a(b,_ 1277). 


We can compute ab using this equation. We first note that ab; = a if b, = 1 and ab; = 0 if b, = 0. 
Each time we multiply a term by 2, we shift its binary expansion one place to the left and add a 
zero at the tail end of the expansion. Consequently, we can obtain (ab;)2/ by shifting the binary 
expansion of ab, j places to the left, adding j zero bits at the tail end of this binary expansion. 


Finally, we obtain ab by adding the n integers ab,2/, j=0,1,2,...,n— 1. 
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110 
x 101 
110 
000 
110 


11110 
FIGURE 2 


Multiplying 
(110), and (101),. 


EXAMPLE 11 


Algorithm 3 displays this procedure for multiplication. 


ALGORITHM 3 Multiplication of Integers. 


procedure multiply(a, b: positive integers) 
{the binary expansions of a and b are (a4,,_,4,,_ ... dj) 
and (b,,_, 5,» .... bibo), respectively } 
for j:=Oton-1 
if b; = 1 then c; := a shifted j places 
else c; := 0 
{co Cy, -<-> Cy, are the partial products} 
p:=0 
for j:=Oton-1 
p := add(p, c;) 
return p {p is the value of ab} 


Example 10 illustrates the use of this algorithm. 


Find the product of a = (110), and b = (101). 


Solution: First note that 
aby <2" = (110), +1 - 2° = (110), 
ab, - 2! = (110), -0- 2! = (0000), 
and 
ab, «27 = (110), «1 +2? = (11000). 


To find the product, add (110),, (0000),, and (11000),. Carrying out these additions (us- 
ing Algorithm 2, including initial zero bits when necessary) shows that ab = (1 1110),. This 
multiplication is displayed in Figure 2. < 


Next, we determine the number of additions of bits and shifts of bits used by Algorithm 3 
to multiply two integers. 


How many additions of bits and shifts of bits are used to multiply a and b using Algorithm 3? 


Solution: Algorithm 3 computes the products of a and b by adding the partial products 
Co Cy, Cz... and c,_,;. When b, = 1, we compute the partial product c, by shifting the binary 
expansion of a by j bits. When b, = 0, no shifts are required because c, = 0. Hence, to find all 
n of the integers ab,2/, jJ=0,1,...,n— 1, requires at most 


O+14+2+-+-4+n-1 


shifts. Hence, by Example 5 in Section 3.2 the number of shifts required is O(n”). 

To add the integers ab; from j = 0 to j =n — 1 requires the addition of an n-bit integer, 
an (n + 1)-bit integer, ..., and a (2n)-bit integer. We know from Example 9 that each of these 
additions requires O(n) additions of bits. Consequently, a total of O(n’) additions of bits are 
required for all n additions. < 


Surprisingly, there are more efficient algorithms than the conventional algorithm for multi- 
plying integers. One such algorithm, which uses O(n!>*>) bit operations to multiply n-bit num- 
bers, will be described in Section 8.3. 


If you are rusty with the 
laws for exponents, this 
is the time to review 
them! See Theorem 1 in 
Appendix 2. 
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ALGORITHM FOR div AND mod Given integers a and d, d>O, we can find q = 
a div d and r= a mod d using Algorithm 4. In this brute-force algorithm, when a is positive 
we subtract d from a as many times as necessary until what is left is less than d. The number of 
times we perform this subtraction is the quotient and what is left over after all these subtractions 
is the remainder. Algorithm 4 also covers the case where a is negative. This algorithm finds the 
quotient q and remainder r when |a| is divided by d. Then, when a < 0 and r > 0, it uses these to 
find the quotient —(g + 1) and remainder d — r when a is divided by d. We leave it to the reader 
(Exercise 65) to show that, assuming that a > d, this algorithm uses O(q log a) bit operations. 


ALGORITHM 4 Computing div and mod. 


procedure division algorithm(a: integer, d: positive integer) 


q:=0 

r:=|a| 

while r > d 
r:=r-d 
q:=q+1 

if a < 0 and r > 0 then 
ri=d-r 
q:= -q+ 1) 


return (q, r) {q = a div d is the quotient, r = a mod d is the remainder } 


There are more efficient algorithms than Algorithm 4 for determining the quotient q = 
a div d and the remainder r = a mod d when a positive integer a is divided by a positive integer 
d (see [Kn98] for details). These algorithms require O(log a - log d) bit operations. If both of 
the binary expansions of a and d contain n or fewer bits, then we can replace loga - logd by 
n?. This means that we need O(n”) bit operations to find the quotient and remainder when a is 
divided by d. 


4.2.4 Modular Exponentiation 


In cryptography it is important to be able to find b” mod m efficiently without using an excessive 
amount of memory, where b, n, and m are large integers. It is impractical to first compute b” 
and then find its remainder when divided by m, because b” can be a huge number and we will 
need a huge amount of computer memory to store such numbers. Instead, we can avoid time and 
memory problems by using an algorithm that employs the binary expansion of the exponent n. 

Before we present an algorithm for fast modular exponentiation based on the binary ex- 
pansion of the exponent, first observe that we can avoid using large amount of memory if we 
compute b” mod m by successively computing b* mod m for k = 1, 2, ..., n using the fact that 
b*+! mod m = b(b‘ mod m) mod m (by Corollary 2 of Theorem 5 of Section 4.1). (Recall that 
1 < b < m.) However, this approach is impractical because it requires n — 1 multiplications of 
integers and n might be huge. 

To motivate the fast modular exponentiation algorithm, we illustrate its basic idea. We will 
explain how to use the binary expansion of n, say n = (a}_ı ... 4,dy)9, to compute b”. First, 
note that 


b" = paar 24 +a) 2+a9 = peeve TOR pa2 . b”, 


This shows that to compute b”, we need only compute the values of b, b?, (b?)? = bt, (b+)? = 
b8, ..., b”. Once we have these values, we multiply the terms b” in this list, where a; = 1. (For 
efficiency and to reduce space requirements, after multiplying by each term, we reduce the result 
modulo m.) 
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È This gives us b”. For example, to compute 3!! we first note that 11 = (1011), so that 3!! = 
38323!, By successively squaring, we find that 3? = 9, 3+ = 9? = 81, and 38 = (81)? = 6561. 

Note that 22"? =p"! Consequently, 31! = 38373! = 6561 - 9 - 3 = 177,147. 

when n is a nonnegative 

integer. 

The algorithm successively finds b mod m, b? mod m, b* mod m,..., b% 'mod m 
and multiplies together those terms b” mod m where a;=1, finding the remainder 
of the product when divided by m after each multiplication. Note that we need only perform 
O(log,(n)) multiplications. Pseudocode for this algorithm is shown in Algorithm 5. Note that 
in Algorithm 5 we can use the most efficient algorithm available to compute values of the mod 


function, not necessarily Algorithm 4. 


Be sure to reduce 
modulo m after each 
multiplication! 


ALGORITHM 5 Fast Modular Exponentiation. 


procedure modular exponentiation(b: integer, n = (a,_,d,_7 ... 4,49); 
m: positive integers) 
x:=1 
power := bmod m 
for i := 0tok-— 1 
if a; = 1 then x := (x- power) mod m 
power := (power - power) mod m 
return x{x equals b” mod m} 


We illustrate how Algorithm 5 works in Example 12. 
EXAMPLE 12 Use Algorithm 5 to find 36^ mod 645. 


Solution: Algorithm 5 initially sets x = 1 and power = 3 mod 645 = 3. In the computation of 
3644 mod 645, this algorithm determines 3” mod 645 for j = 1, 2, ... , 9 by successively squaring 
and reducing modulo 645. Ifa; = 1 (where a, is the bit in the jth position in the binary expansion 
of 644, which is (1010000100), ), it multiplies the current value of x by 3” mod 645 and reduces 
the result modulo 645. Here are the steps used: 


0: Because ay = 0, we have x = 1 and power = 3* mod 645 = 9 mod 645 = 9; 
1: Because a, = 0, we have x = 1 and power = 9? mod 645 = 81 mod 645 = 81; 
2: Because a, = 1, we have x = 1 - 81 mod 645 = 81 and power = 812 mod 645 = 6561 mod 645 = 111; 
3: Because a, = 0, we have x = 81 and power = 1112 mod 645 = 12,321 mod 645 = 66; 
i = 4: Because a, = 0, we have x = 81 and power = 662 mod 645 = 4356 mod 645 = 486; 
5: Because a; = 0, we have x = 81 and power = 4862 mod 645 = 236,196 mod 645 = 126; 
6: Because a, = 0, we have x = 81 and power = 126? mod 645 = 15,876 mod 645 = 396; 
7 


: Because a, = 1, we find that x = (81 - 396) mod 645 = 471 and power = 396? mod 645 = 156,816 
mod 645 = 81; 


i = 8: Because ag = 0, we have x = 471 and power = 81° mod 645 = 6561 mod 645 = 111; 
i = 9: Because dy = 1, we find that x = (471 - 111) mod 645 = 36. 


This shows that following the steps of Algorithm 5 produces the result 3% mod 645 = 36. 
< 


Algorithm 5 is quite efficient; it uses O((log m}? logn) bit operations to find b” mod m (see 
Exercise 64). 


Exercises 


12. 


13. 


14. 


15. 


. Convert the decimal expansion of each of these integers 


to a binary expansion. 


a) 231 b) 4532 c) 97644 


. Convert the decimal expansion of each of these integers 


to a binary expansion. 


a) 321 b) 1023 c) 100632 


. Convert the binary expansion of each of these integers to 


a decimal expansion. 


a) (11111), 
c) (10101 0101), 


b) (10 0000 0001), 
d) (110 1001 0001 0000), 


. Convert the binary expansion of each of these integers to 


a decimal expansion. 


a) (1 1011), 
c) (11 1011 1110), 


b) (10 1011 0101), 
d) (111 11000001 1111), 


. Convert the octal expansion of each of these integers to 


a binary expansion. 
a) (572)g 
c) (423). 


b) (1604), 
d) (2417), 


. Convert the binary expansion of each of these integers to 


an octal expansion. 

a) (1111 0111), 

b) (1010 1010 1010), 

c) (111 0111 0111 0111), 
d) (101 0101 0101 0101), 


. Convert the hexadecimal expansion of each of these in- 


tegers to a binary expansion. 


a) (80E)j¢ b) (135AB)j6 
c) (ABBA) 6 d) (DEFACED), 
. Convert (BADFACED),,¢ from its hexadecimal expan- 


sion to its binary expansion. 


. Convert (ABCDEF),, from its hexadecimal expansion to 


its binary expansion. 


. Convert each of the integers in Exercise 6 from a binary 


expansion to a hexadecimal expansion. 


. Convert (1011 0111 1011), from its binary expansion to 


its hexadecimal expansion. 

Convert (1 1000 0110 0011), from its binary expansion 
to its hexadecimal expansion. 

Show that the hexadecimal expansion of a positive inte- 
ger can be obtained from its binary expansion by group- 
ing together blocks of four binary digits, adding initial 
zeros if necessary, and translating each block of four 
binary digits into a single hexadecimal digit. 

Show that the binary expansion of a positive integer can 
be obtained from its hexadecimal expansion by translat- 
ing each hexadecimal digit into a block of four binary 
digits. 

Show that the octal expansion of a positive integer can be 
obtained from its binary expansion by grouping together 
blocks of three binary digits, adding initial zeros if nec- 
essary, and translating each block of three binary digits 
into a single octal digit. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 
26. 
27. 
28. 
29. 


30. 
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Show that the binary expansion of a positive integer can 
be obtained from its octal expansion by translating each 
octal digit into a block of three binary digits. 

Convert (7345321) to its binary expansion and 
(10 1011 1011), to its octal expansion. 

Give a procedure for converting from the hexadecimal 
expansion of an integer to its octal expansion using bi- 
nary notation as an intermediate step. 

Give a procedure for converting from the octal expansion 
of an integer to its hexadecimal expansion using binary 
notation as an intermediate step. 

Explain how to convert from binary to base 64 expan- 
sions and from base 64 expansions to binary expansions 
and from octal to base 64 expansions and from base 64 
expansions to octal expansions. 

Find the sum and the product of each of these pairs of 
numbers. Express your answers as a binary expansion. 

a) (1000111), (111 0111), 

b) (1110 1111),, (1011 1101), 

c) (10 1010 1010),, (1 1111 0000), 

d) (10 0000 0001),, (11 1111 1111), 

Find the sum and product of each of these pairs of num- 
bers. Express your answers as a base 3 expansion. 

a) (112),, (210), 

b) (2112), (12021), 

c) (20001)3, (1111); 

d) (120021),, (2002), 

Find the sum and product of each of these pairs of num- 
bers. Express your answers as an octal expansion. 

a) (763)<., (147); 

b) (6001)g, (272)g 

c) (1111), (777). 

d) (54321)g, (3456). 

Find the sum and product of each of these pairs of num- 
bers. Express your answers as a hexadecimal expan- 
sion. 

a) (LAE),,, (BBO), 

b) (20CBA),,, (A01),¢ 

c) (ABCDE),,. (111 1))i¢ 

d) (E0000E); 6, (BAAA);6 

Use Algorithm 5 to find 76^ mod 645. 

Use Algorithm 5 to find 11° mod 645. 

Use Algorithm 5 to find 3%% mod 99. 

Use Algorithm 5 to find 123!%! mod 101. 

Show that every positive integer can be represented 
uniquely as the sum of distinct powers of 2. [Hint: Con- 
sider binary expansions of integers. ] 

It can be shown that every integer can be uniquely repre- 
sented in the form 


e,3* +e, 3h 1 +- + e3 + ep, 


where e= —1, 0, or 1 for j = 0, 1, 2, ..., k. Expansions of 
this type are called balanced ternary expansions. Find 
the balanced ternary expansions of 


a) 5. b) 13. c) 37. d) 79. 
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31. Show that a positive integer is divisible by 3 if and only 
if the sum of its decimal digits is divisible by 3. 


32. Show that a positive integer is divisible by 11 if and only 
if the difference of the sum of its decimal digits in even- 
numbered positions and the sum of its decimal digits in 
odd-numbered positions is divisible by 11. 


33. Show that a positive integer is divisible by 3 if and only 
if the difference of the sum of its binary digits in even- 
numbered positions and the sum of its binary digits in 
odd-numbered positions is divisible by 3. 

34. Determine how we can use the decimal expansion of an 
integer n to determine whether n is divisible by 
a) 2 b) 5 c) 10 

35. Determine how we can use the decimal expansion of an 
integer n to determine whether n is divisible by 
a) 4 b) 25 c) 20 

36. Suppose that n and b are positive integers with b > 2 and 
the base b expansion of n is n = (a,,a .. a,d),. Find 
the base b expansion of 
a) bn. b) bn; 

c) [n/b], d) |n/b?]. 

37. Prove that if n and b are positive integers with b > 2 the 

base b representation of n has |log,n] + 1 digits. 


m-l" 


38. Find the decimal expansion of the number with the n- 
digit base seven expansion (111... 111), (with n 1’s). 
[Hint: Use the formula for the sum of the terms of a geo- 
metric progression. ] 


39. Find the decimal expansion of the number with the 37 
bit binary expansion (101101....101101), (so that the bi- 
nary expansion is made of n copies of 101). [Hint: Use 
the formula for the sum of the terms of a geometric pro- 
gression. ] 


One’s complement representations of integers are used to 
simplify computer arithmetic. To represent positive and nega- 
tive integers with absolute value less than 2"-1 a total of n bits 
is used. The leftmost bit is used to represent the sign. A 0 bit 
in this position is used for positive integers, and a | bit in this 
position is used for negative integers. For positive integers, 
the remaining bits are identical to the binary expansion of the 
integer. For negative integers, the remaining bits are obtained 
by first finding the binary expansion of the absolute value of 
the integer, and then taking the complement of each of these 
bits, where the complement of a 1 is a 0 and the complement 
of aOisal. 


40. Find the one’s complement representations, using bit 
strings of length six, of the following integers. 
a) 22 b) 31 c) -7 d) -19 

41. What integer does each of the following one’s comple- 
ment representations of length five represent? 


a) 11001 b) 01101 
c) 10001 d) 11111 
42. If m is a positive integer less than 2”7!, how is the 


one’s complement representation of —m obtained from 
the one’s complement of m, when bit strings of length n 
are used? 


43. How is the one’s complement representation of the sum 
of two integers obtained from the one’s complement rep- 
resentations of these integers? 


44. How is the one’s complement representation of the differ- 
ence of two integers obtained from the one’s complement 
representations of these integers? 


45. Show that the integer m with one’s complement represen- 

tation (a,,_;4,,_) ... @;d9) can be found using the equation 

m = —a,_\(2"! — 1) +a, 92”? + ++ + ay -2 + ap. 
Two’s complement representations of integers are also used 
to simplify computer arithmetic and are used more commonly 
than one’s complement representations. To represent an inte- 
ger x with —2”"! < x < 2""! — 1 for a specified positive in- 
teger n, a total of n bits is used. The leftmost bit is used to 
represent the sign. A 0 bit in this position is used for positive 
integers, and a | bit in this position is used for negative inte- 
gers, just as in one’s complement expansions. For a positive 
integer, the remaining bits are identical to the binary expan- 
sion of the integer. For a negative integer, the remaining bits 
are the bits of the binary expansion of 2”~! — |x|. Two’s com- 
plement expansions of integers are often used by computers 
because addition and subtraction of integers can be performed 
easily using these expansions, where these integers can be ei- 
ther positive or negative. 


46. Answer Exercise 40, but this time find the two’s comple- 
ment expansion using bit strings of length six. 


47. Answer Exercise 41 if each expansion is a two’s comple- 
ment expansion of length five. 


48. Answer Exercise 42 for two’s complement expansions. 
49. Answer Exercise 43 for two’s complement expansions. 
50. Answer Exercise 44 for two’s complement expansions. 


51. Show that the integer m with two’s complement represen- 
tation (a„—14,„—2 ... @;dg) can be found using the equation 
m=—a,_;° 2"! +a, 52"? +++ +a, -2+ a. 


52. Give a simple algorithm for forming the two’s comple- 
ment representation of an integer from its one’s comple- 
ment representation. 


53. Sometimes integers are encoded by using four-digit bi- 
nary expansions to represent each decimal digit. This pro- 
duces the binary coded decimal form of the integer. For 
instance, 791 is encoded in this way by 011110010001. 
How many bits are required to represent a number with n 
decimal digits using this type of encoding? 


A Cantor expansion is a sum of the form 
a,n! +a, (n 1)! + +a,2!+a,1!, 


where a; is an integer with 0 < a; < i for i = 1, 2,...,n. 
54. Find the Cantor expansions of 

a) 2. b) 7. 

c) 19. d) 87. 

e) 1000. f) 1,000,000. 


= 55: 


* 56. 


57. 


58. 


59. 


60. 


Describe an algorithm that finds the Cantor expansion of 
an integer. 


Describe an algorithm to add two integers from their Can- 
tor expansions. 


Add (10111), and (11010), by working through each step 
of the algorithm for addition given in the text. 

Multiply (1110), and (1010), by working through each 
step of the algorithm for multiplication given in the text. 
Describe an algorithm for finding the difference of two 
binary expansions. 


Estimate the number of bit operations used to subtract 
two binary expansions. 


61. 


62. 


63. 


* 64, 


65. 
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Devise an algorithm that, given the binary expansions of 
the integers a and b, determines whether a > b, a = b, or 
a<b. 

How many bit operations does the comparison algo- 
rithm from Exercise 61 use when the larger of a and b 
has n bits in its binary expansion? 

Estimate the complexity of Algorithm 1 for finding the 
base b expansion of an integer n in terms of the number 
of divisions used. 

Show that Algorithm 5 uses O((log m}? log n) bit opera- 
tions to find b” mod m. 

Show that Algorithm 4 uses O(q log a) bit operations, as- 
suming that a > d. 


Primes and Greatest Common Divisors 


Definition 1 


4.3.1 Introduction 


In Section 4.1 we studied the concept of divisibility of integers. One important concept based 
on divisibility is that of a prime number. A prime is an integer greater than 1 that is divisible 
by no positive integers other than 1 and itself. The study of prime numbers goes back to ancient 
times. Thousands of years ago it was known that there are infinitely many primes; the proof of 
this fact, found in the works of Euclid, is famous for its elegance and beauty. 

We will discuss the distribution of primes among the integers. We will describe some of 
the results about primes found by mathematicians in the last 400 years. In particular, we will 
introduce an important theorem, the fundamental theorem of arithmetic. This theorem, which 
asserts that every positive integer can be written uniquely as the product of primes in nonde- 
creasing order, has many interesting consequences. We will also discuss some of the many old 
conjectures about primes that remain unsettled today. 

Primes have become essential in modern cryptographic systems, and we will develop some 
of their properties important in cryptography. For example, finding large primes is essential 
in modern cryptography. The length of time required to factor large integers into their prime 
factors is the basis for the strength of some important modern cryptographic systems. 

In this section we will also study the greatest common divisor of two integers, as well as the 
least common multiple of two integers. We will develop an important algorithm for computing 
greatest common divisors, called the Euclidean algorithm. 


4.3.2 Primes 


Every integer greater than 1 is divisible by at least two integers, because a positive integer is 
divisible by 1 and by itself. Positive integers that have exactly two different positive integer 
factors are called primes. 


An integer p greater than 1 is called prime if the only positive factors of p are 1 and p. 
A positive integer that is greater than 1 and is not prime is called composite. 


Remark: The integer 1 is not prime, because it has only one positive factor. Note also that an 
integer n is composite if and only if there exists an integer a such that a | nand 1<a<n. 
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EXAMPLE 1 


THEOREM 1 


EXAMPLE 2 


Extra » 
Examples 


THEOREM 2 


The integer 7 is prime because its only positive factors are 1 and 7, whereas the integer 9 is 
composite because it is divisible by 3. < 


The primes are the building blocks of positive integers, as the fundamental theorem of 
arithmetic shows. The proof will be given in Section 5.2. 


THE FUNDAMENTAL THEOREM OF ARITHMETIC Every integer greater than 1 
can be written uniquely as a prime or as the product of two or more primes, where the prime 
factors are written in order of nondecreasing size. 


Example 2 gives some prime factorizations of integers. 


The prime factorizations of 100, 641, 999, and 1024 are given by 


100 =2-2-5-5 = 2257, 


641 = 641, 
999 = 3-3-3-37 =3°- 37, 
1024 =2-2.2-2.2.-2.2.2.2.2 =D", < 


4.3.3 Trial Division 


It is often important to show that a given integer is prime. For instance, in cryptology, large 
primes are used in some methods for making messages secret. One procedure for showing that 
an integer is prime is based on the following observation. 


If n is a composite integer, then n has a prime divisor less than or equal to s/n. 


Proof: If n is composite, by the definition of a composite integer, we know that it has a factor 
a with 1 <a <n. Hence, by the definition of a factor of a positive integer, we have n = ab, 
where b is a positive integer greater than 1. We will show that a < s/n orb< s/n. Ifa> s/n 
and b> s/n, then ab > s/n . yn =n, which is a contradiction. Consequently, a < s/n or 
b< y/n. Because both a and b are divisors of n, we see that n has a positive divisor not 


exceeding y/n. This divisor is either prime or, by the fundamental theorem of arithmetic, 
has a prime divisor less than itself. In either case, n has a prime divisor less than or equal 


to y/n. < 


From Theorem 2, it follows that an integer is prime if it is not divisible by any prime less 
than or equal to its square root. This leads to the brute-force algorithm known as trial division. 
To use trial division we divide n by all primes not exceeding yn and conclude that n is prime 
if it is not divisible by any of these primes. In Example 3 we use trial division to show that 101 
is prime. 


EXAMPLE 3 


EXAMPLE 4 


Links » 
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Show that 101 is prime. 


Solution: The only primes not exceeding 101 are 2, 3, 5, and 7. Because 101 is not divisible 
by 2, 3, 5, or 7 (the quotient of 101 and each of these integers is not an integer), it follows that 
101 is prime. < 


Because every integer has a prime factorization, it would be useful to have a procedure for 
finding this prime factorization. Consider the problem of finding the prime factorization of n. 
Begin by dividing n by successive primes, starting with the smallest prime, 2. If n has a prime 
factor, then by Theorem 3 a prime factor p not exceeding s/n will be found. So, if no prime factor 


not exceeding Jn is found, then n is prime. Otherwise, if a prime factor p is found, continue 
by factoring n/p. Note that n/p has no prime factors less than p. Again, if n/p has no prime 
factor greater than or equal to p and not exceeding its square root, then it is prime. Otherwise, 
if it has a prime factor q, continue by factoring n/(pq). This procedure is continued until the 
factorization has been reduced to a prime. This procedure is illustrated in Example 4. 


Find the prime factorization of 7007. 


Solution: To find the prime factorization of 7007, first perform divisions of 7007 by succes- 
sive primes, beginning with 2. None of the primes 2, 3, and 5 divides 7007. However, 7 di- 
vides 7007, with 7007/7 = 1001. Next, divide 1001 by successive primes, beginning with 7. 
It is immediately seen that 7 also divides 1001, because 1001/7 = 143. Continue by divid- 
ing 143 by successive primes, beginning with 7. Although 7 does not divide 143, 11 does di- 
vide 143, and 143/11 = 13. Because 13 is prime, the procedure is completed. It follows that 
7007 = 7-1001 =7-7-143 =7-7- 11-13. Consequently, the prime factorization of 7007 is 
7-7-11-13=7°-11-13. < 


Prime numbers were studied in ancient times for philosophical reasons. Today, there are 
highly practical reasons for their study. In particular, large primes play a crucial role in cryp- 
tography, as we will see in Section 4.6. 


4.3.4 The Sieve of Eratosthenes 


Note that composite integers not exceeding 100 must have a prime factor not exceeding 10. 
Because the only primes less than 10 are 2, 3, 5, and 7, the primes not exceeding 100 are these 
four primes and those positive integers greater than 1 and not exceeding 100 that are divisible 
by none of 2, 3, 5, or 7. 

The sieve of Eratosthenes is used to find all primes not exceeding a specified positive 
integer. For instance, the following procedure is used to find the primes not exceeding 100. We 
begin with the list of all integers between 1 and 100. To begin the sieving process, the integers 
that are divisible by 2, other than 2, are deleted. Because 3 is the first integer greater than 2 that 
is left, all those integers divisible by 3, other than 3, are deleted. Because 5 is the next integer 
left after 3, those integers divisible by 5, other than 5, are deleted. The next integer left is 7, 


| ERATOSTHENES (276 B.C.E.—194 B.C.E.) It is known that Eratosthenes was born in Cyrene, a Greek colony 
‘| west of Egypt, and spent time studying at Plato’s Academy in Athens. We also know that King Ptolemy II invited 


| Eratosthenes to Alexandria to tutor his son and that later Eratosthenes became chief librarian at the famous 


| library at Alexandria, a central repository of ancient wisdom. Eratosthenes was an extremely versatile scholar, 


writing on mathematics, geography, astronomy, history, philosophy, and literary criticism. Besides his work in 


i mathematics, he is most noted for his chronology of ancient history and for his famous measurement of the size 


i S| of the earth. 


Source: Math Tutor Archive 
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TABLE 1 The Sieve of Eratosthenes. 


Integers divisible by 2 other than 2 Integers divisible by 3 other than 3 
receive an underline. receive an underline. 
1 2 3 4 

13 

23 

33 

43 

53 

63 

73 

83 

93 


Integers divisible by 5 other than 5 Integers divisible by 7 other than 7 receive 


receive an underline. an underline; integers in color are prime. 


¢ AS 6 F7 
16 
26 


IS WSUSiSMS USNS WSU 


so those integers divisible by 7, other than 7, are deleted. Because all composite integers not 
exceeding 100 are divisible by 2, 3, 5, or 7, all remaining integers except 1 are prime. In Table 1, 
the panels display those integers deleted at each stage, where each integer divisible by 2, other 
than 2, is underlined in the first panel, each integer divisible by 3, other than 3, is underlined 
in the second panel, each integer divisible by 5, other than 5, is underlined in the third panel, 
and each integer divisible by 7, other than 7, is underlined in the fourth panel. The integers not 
underlined are the primes not exceeding 100. We conclude that the primes less than 100 are 2, 
3,5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, and 97. 


THE INFINITUDE OF PRIMES Ithas long been known that there are infinitely many primes. 
This means that whenever p,, p>, ...,p, are the n smallest primes, we know there is a larger 
prime not listed. We will prove this fact using a proof given by Euclid in his famous mathematics 
text, The Elements. This simple, yet elegant, proof is considered by many mathematicians to be 
among the most beautiful proofs in mathematics. It is the first proof presented in the book Proofs 
from THE BOOK (AiZi[14]), where THE BOOK refers to the imagined collection of perfect 
proofs that the legendary mathematician Paul Erdős claimed is maintained by God. By the way, 
there are a vast number of different proofs that there are an infinitude of primes, and new ones 
are published surprisingly frequently. 


THEOREM 3 


? 


EXAMPLE 5 


@©Apic/Getty Images 
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There are infinitely many primes. 


Proof: We will prove this theorem using a proof by contradiction. We assume that there are only 
finitely many primes, p4, P, ...,P,- Let 


Q=p\p.°"p, +1. 


By the fundamental theorem of arithmetic, Q is prime or else it can be written as the prod- 
uct of two or more primes. However, none of the primes p; divides Q, for if p; | Q, then p; 
divides Q — p,p, + P, = 1. Hence, there is a prime not in the list p4, po, ..., Pa. This prime is 
either Q, if it is prime, or a prime factor of Q. This is a contradiction because we assumed that 
we have listed all the primes. Consequently, there are infinitely many primes. < 


Remark: Note that in this proof we do not state that Q is prime! Furthermore, in this proof, we 
have given a nonconstructive existence proof that given any n primes, there is a prime not in 
this list. For this proof to be constructive, we would have had to explicitly give a prime not in 
our original list of n primes. 


Because there are infinitely many primes, given any positive integer there are primes greater 
than this integer. There is an ongoing quest to discover larger and larger prime numbers; for al- 
most all the last 300 years, the largest prime known has been an integer of the special form 
2P — 1, where p is also prime. (Note that 2” — 1 cannot be prime when n is not prime; see 
Exercise 9.) Such primes are called Mersenne primes, after the French monk Marin Mersenne, 
who studied them in the seventeenth century. The reason that the largest known prime has usu- 
ally been a Mersenne prime is that there is an extremely efficient test, known as the Lucas— 
Lehmer test, for determining whether 2? — 1 is prime. Furthermore, it is not currently possible 
to test numbers not of this or certain other special forms anywhere near as quickly to determine 
whether they are prime. 


The numbers 2? — 1 = 3,2? — 1 = 7,25 — 1 = 31 and 27” — 1 = 127 are Mersenne primes, while 
2!! — 1 = 2047 is not a Mersenne prime because 2047 = 23 - 89. < 


Progress in finding Mersenne primes has been steady since computers were invented. As of 
early 2018, 50 Mersenne primes were known, with 19 found since 1990. The largest Mersenne 
prime known (again as of early 2018) is 277232917 — 1, a number with 23,249,425 decimal 


MARIN MERSENNE (1588-1648) Mersenne was born in Maine, France, into a family of laborers and 
attended the College of Mans and the Jesuit College at La Fléche. He continued his education at the Sor- 
bonne, studying theology from 1609 to 1611. He joined the religious order of the Minims in 1611, a 
group whose name comes from the word minimi (the members of this group were extremely humble; they 
considered themselves the least of all religious orders). Besides prayer, the members of this group de- 
voted their energy to scholarship and study. In 1612 he became a priest at the Place Royale in Paris; be- 
tween 1614 and 1618 he taught philosophy at the Minim Convent at Nevers. He returned to Paris in 1619, 
where his cell in the Minims de l’Annociade became a place for meetings of French scientists, philoso- 
phers, and mathematicians, including Fermat and Pascal. Mersenne corresponded extensively with schol- 
ars throughout Europe, serving as a clearinghouse for mathematical and scientific knowledge, a function 
later served by mathematical journals (and today also by the Internet). Mersenne books covering mechanics, 


wrote mathematical physics, mathematics, music, and acoustics. He studied prime numbers and tried unsuccessfully to construct a 
formula representing all primes. In 1644 Mersenne claimed that 2? — 1 is prime for p = 2, 3, 5, 7, 13, 17, 19, 31, 67, 127, 257 but 
is composite for all other primes less than 257. It took over 300 years to determine that Mersenne’s claim was wrong five times. 
Specifically, 2? — 1 is not prime for p = 67 and p = 257 but is prime for p = 61, p = 87, and p = 107. It is also noteworthy that 
Mersenne defended two of the most famous men of his time, Descartes and Galileo, from religious critics. He also helped expose 
alchemists and astrologers as frauds. 
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Links > 


digits, which was shown to be prime in December, 2017. A communal effort, the Great In- 
ternet Mersenne Prime Search (GIMPS), is devoted to the search for new Mersenne primes. 
You can join this search, and if you are lucky, find a new Mersenne prime and possibly even 
win a cash prize. By the way, even the search for Mersenne primes has practical implications. 
A commonly used quality control test for supercomputers is to replicate the Lucas—Lehmer test 
that establishes the primality of a large Mersenne prime. Also, in January, 2016, it was reported 
that a bug in the Intel Skylake processor was found when GIMPS software was run. (See [Ro10] 
for more information about the quest for finding Mersenne primes.) 


THE DISTRIBUTION OF PRIMES Theorem 3 tells us that there are infinitely many primes. 
However, how many primes are less than a positive number x? This question interested mathe- 
maticians for many years; in the late eighteenth century, mathematicians produced large tables 
of prime numbers to gather evidence concerning the distribution of primes. Using this evi- 
dence, the great mathematicians of the day, including Gauss and Legendre, conjectured, but did 
not prove, Theorem 4. 


THE PRIME NUMBER THEOREM Theratio of z(x), the number of primes not exceed- 
ing x, and x/ ln x approaches 1 as x grows without bound. (Here In x is the natural logarithm 
of x.) 


The prime number theorem was first proved in 1896 by the French mathematician Jacques 
Hadamard and the Belgian mathematician Charles-Jean-Gustave-Nicholas de la Vallée-Poussin 
using the theory of complex variables. Although proofs not using complex variables have been 
found, all known proofs of the prime number theorem are quite complicated. Many refinements 
of the prime number theorem have been proved, with many addressing the error made by esti- 
mating z(x) with x/ In x, and by estimating z(x) with other functions. Many unsolved questions 
remain in this area of study. 

Table 2 displays z(x), x/ ln x, and their ratio, for x = 10” where 3 < n < 10. A tremendous 
amount of effort has been devoted to computing z(x) for progressively larger values of x. As of 
late 2017, the number of primes less than or equal to 10” has been determined for all positive 
integers n with n < 26. In particular, it is known that 


n(10*°) = 1,699,246, 750,872,437, 141,327,603, 
to the nearest integer 


n(10*°) — (107 / In 107°) = 28,883,358, 936,853, 188,823,261, 


TABLE 2 Approximating (x) by x/In x. 


ron 
10° 168 144.8 
1229 1085.7 
9592 8685.9 


78,498 72,382.4 
664,579 620,420.7 
5,761,455 5,428,681.0 
50,847,534 48,254,942.4 
455,052,512 434,294,481.9 
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and up to six decimal places 
(107°) /(10°/ In (107°)) = 1.01729. 


You can find a great deal of computational data relating to z(x) and functions that estimate z(x) 
using the web. 

We can use the prime number theorem to estimate the probability that a randomly chosen 
number is prime. (See Chapter 7 to learn the basics of probability theory.) The prime number 
theorem tells us that the number of primes not exceeding x can be approximated by x/Inx. 
Consequently, the odds that a randomly selected positive integer less than n is prime are ap- 
proximately (n/Inn)/n = 1/1nn. Sometimes we need to find a prime with a particular number 
of digits. We would like an estimate of how many integers with a particular number of digits 
we need to select before we encounter a prime. Using the prime number theorem and calculus, 
it can be shown that the probability that an integer n is prime is also approximately 1/ 1n n. For 
example, the odds that an integer near 1010% is prime are approximately 1/ 1n 10!°, which is 
approximately 1/2300. (Note that if we choose only odd numbers, we double our chances of 
finding a prime.) 

Using trial division with Theorem 2 gives procedures for factoring and for primality testing. 
However, these procedures are not efficient algorithms; many much more practical and efficient 
algorithms for these tasks have been developed. Factoring and primality testing have become 
important in the applications of number theory to cryptography. This has led to a great interest 
in developing efficient algorithms for both tasks. Clever procedures have been devised in the 
last 30 years for efficiently generating large primes. Moreover, in 2002, an important theoretical 
discovery was made by Manindra Agrawal, Neeraj Kayal, and Nitin Saxena. They showed there 
is a polynomial-time algorithm in the number of bits in the binary expansion of an integer for 
determining whether a positive integer is prime. Algorithms based on their work use O((log n)°) 
bit operations to determine whether a positive integer n is prime. 

However, even though powerful new factorization methods have been developed in the 
same time frame, factoring large numbers remains extraordinarily more time-consuming than 
primality testing. No polynomial-time algorithm for factoring integers is known. Nevertheless, 
the challenge of factoring large numbers interests many people. There is a communal effort on 
the Internet to factor large numbers, especially those of the special form k” + 1, where k is a 
small positive integer and n is a large positive integer (such numbers are called Cunningham 
numbers). At any given time, there is a list of the “Ten Most Wanted” large numbers of this type 
awaiting factorization. 


CHARLES-JEAN-GUSTAVE-NICHOLAS DE LA VALLEE-POUSSIN (1866-1962) De la Vallée-Poussin, 
the son of a professor of geology, was born in Louvain, Belgium. He attended the Jesuit College at Mons, first 
studying philosophy, but then turning to engineering. After graduating, he devoted himself to mathematics 
instead of engineering. His most important contribution to mathematics was his proof of the prime number 
theorem. He also established results about the distribution of primes in arithmetic progressions and refined 
the prime number theorem to include error estimates. De la Vallée-Poussin made important contributions to 
differential equations, analysis, and approximation theory. He also wrote a textbook, Cours d’analyse, which 
had significant impact on mathematical thought in the first half of the twentieth century. 


JACQUES HADAMARD (1865-1963) Hadamard, whose father was a Latin teacher and mother a distin- 
guished piano teacher, was born in Versailles, France. After graduating from college, he taught at a secondary 
school in Paris. After receiving his Ph.D. in 1892, he was a lecturer at the Faculté des Sciences of Bordeaux. 
Later, he served on the faculties of the Sorbonne, the Collége de France, the Ecole Polytéchnique, and the Ecole 
Centrale des Arts et Manufacturers. Hadamard made significant contributions to complex analysis, functional 
analysis, and mathematical physics. He was recognized as an innovative teacher, writing many articles about 
elementary mathematics that were used in French schools and a widely used elementary geometry book. 
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PRIMES AND ARITHMETIC PROGRESSIONS Every odd integer is in one of the two arith- 
metic progressions 4k + 1 or 4k +3, k = 1,2,.... Because we know that there are infinitely 
many primes, we can ask whether there are infinitely many primes in both of these arithmetic 
progressions. The primes 5, 13, 17, 29, 37, 41, ... are in the arithmetic progression 4k + 1; the 
primes 3, 7, 11, 19, 23, 31, 43, ... are in the arithmetic progression 4k + 3. Looking at the evi- 
dence hints that there may be infinitely many primes in both progressions. What about other 
arithmetic progressions ak + b, k = 1, 2, ..., where no integer greater than one divides both a 
and b? Do they contain infinitely many primes? The answer was provided by the German math- 
ematician G. Lejeune Dirichlet, who proved that every such arithmetic progression contains 
infinitely many primes. His proof, and all proofs found later, are beyond the scope of this book. 
However, it is possible to prove special cases of Dirichlet’s theorem using the ideas developed 
in this book. For example, Exercises 54 and 55 ask for proofs that there are infinitely many 
primes in the arithmetic progressions 3k + 2 and 4k + 3, where k is a positive integer. (The hint 
for each of these exercises supplies the basic idea needed for the proof.) 

We have explained that every arithmetic progression ak +b, k = 1, 2, ..., where a and b 
have no common factor greater than one, contains infinitely many primes. But are there long 
arithmetic progressions made up of just primes? For example, some exploration shows that 5, 11, 
17, 23, 29 is an arithmetic progression of five primes and 199, 409, 619, 829, 1039, 1249, 1459, 
1669, 1879, 2089 is an arithmetic progression of ten primes. In the 1930s, the legendary and 
prolific mathematician Paul Erdős conjectured that for every positive integer n greater than two, 
there is an arithmetic progression of length n made up entirely of primes. In 2006, Ben Green 
and Terence Tao were able to prove this conjecture. Their proof, considered to be a mathematical 
tour de force, is a nonconstructive proof that combines powerful ideas from several advanced 
areas of mathematics. 


4.3.5 Conjectures and Open Problems About Primes 


Number theory is noted as a subject for which it is easy to formulate conjectures, some of which 
are difficult to prove and others that remained open problems for many years. We will describe 
some conjectures in number theory and discuss their status in Examples 6-9. 


EXAMPLE 6 It would be useful to have a function f(n) such that f(n) is prime for all positive integers n. If we 
Extra had such a function, we could find large primes for use in cryptography and other applications. 
Examples » 


Looking for such a function, we might check out different polynomial functions, as some 
mathematicians did several hundred years ago. After a lot of computation we may encounter 


TERENCE TAO (BORN 1975) Tao was born in Australia. His father is a pediatrician and his mother taught 
mathematics at a Hong Kong secondary school. Tao was a child prodigy, teaching himself arithmetic at the age 
of two. At 10, he became the youngest contestant at the International Mathematical Olympiad (IMO); he won 
an IMO gold medal at 13. Tao received his bachelor’s and master’s degrees when he was 17, and began graduate 
studies at Princeton, receiving his Ph.D. in three years. In 1996 he became a faculty member at UCLA, where 
he continues to work. 

Tao is extremely versatile; he enjoys working on problems in diverse areas, including harmonic analysis, 
partial differential equations, number theory, and combinatorics. You can follow his work by reading his blog, 
Courtesy of Reed where he discusses progress on various problems. His most famous result is the Green-Tao theorem, which 
Hutchinson/UCLA % 3 $ $ 3 3 $ a : 

says that there are arbitrarily long arithmetic progressions of primes. Tao has made important contributions to 
the applications of mathematics, such as developing a method for reconstructing digital images using the least 
possible amount of information. 

Tao has an amazing reputation among mathematicians; he has become a Mr. Fix-It for researchers in mathematics. The well- 
known mathematician Charles Fefferman, himself a child prodigy, has said that “if you’re stuck on a problem, then one way out is 
to interest Terence Tao.” Tao maintains a popular blog that describes his research work and many mathematical problems in great 
detail. In 2006 Tao was awarded a Fields Medal, the most prestigious award for mathematicians under the age of 40. He was also 
awarded a MacArthur Fellowship in 2006, and in 2008, he received the Allan T. Waterman award, which came with a $500,000 cash 
prize to support research work of scientists early in their careers. Tao’s wife Laura is an engineer at the Jet Propulsion Laboratory. 
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the polynomial f(n) = n? — n + 41. This polynomial has the interesting property that f(n) is 
prime for all positive integers n not exceeding 40. [We have f(1) = 41, f(2) = 43, f(3) = 47, 
(4) = 53, and so on.] This can lead us to the conjecture that f(n) is prime for all positive inte- 
gers n. Can we settle this conjecture? 


Solution: Perhaps not surprisingly, this conjecture turns out to be false; we do not have to look far 
to find a positive integer n for which f(n) is composite, because f(41) = 41? — 41 + 41 = 41?. 
Because f(n) = n? — n + 41 is prime for all positive integers n with 1 < n < 40, we might be 
tempted to find a different polynomial with the property that f (n) is prime for all positive integers 
n. However, there is no such polynomial. It can be shown that for every polynomial f(n) with 
integer coefficients, there is a positive integer y such that f(y) is composite. (See Exercise 23 in 
the Supplementary Exercises.) < 


Many famous problems about primes still await ultimate resolution by clever people. We 
describe a few of the most accessible and better known of these open problems in Examples 7-9. 
Number theory is noted for its wealth of easy-to-understand conjectures that resist attack by all 
but the most sophisticated techniques, or simply resist all attacks. We present these conjectures 
to show that many questions that seem relatively simple remain unsettled even in the twenty-first 
century. 


EXAMPLE 7 Goldbach’s Conjecture In 1742, Christian Goldbach, in a letter to Leonhard Euler, conjec- 
tured that every odd integer n, n > 5, is the sum of three primes. Euler replied that this conjecture 
is equivalent to the conjecture that every even integer n, n > 2, is the sum of two primes (see 
Exercise 21 in the Supplementary Exercises). The conjecture that every even integer n, n > 2, 
is the sum of two primes is now called Goldbach’s conjecture. We can check this conjecture 
for small even numbers. For example, 4 = 2 + 2,6 = 3 + 3, 8 = 5 +3, 10 =7 +3, 12 =7 +5, 

Li > and so on. Goldbach’s conjecture was verified by hand calculations for numbers up to the mil- 
inks ; f : i 
lions prior to the advent of computers. With computers it can be checked for extremely large 
numbers. As of early 2018, the conjecture has been checked for all positive even integers up to 
4 . 10!8. 

Although no proof of Goldbach’s conjecture has been found, most mathematicians believe 
it is true. Several theorems have been proved, using complicated methods from analytic number 
theory far beyond the scope of this book, establishing results weaker than Goldbach’s conjecture. 
Among these are the result that every even integer greater than 2 is the sum of at most six primes 
(proved in 1995 by O. Ramaré) and that every sufficiently large positive integer is the sum of a 
prime and a number that is either prime or the product of two primes (proved in 1966 by J. R. 
Chen). Perhaps Goldbach’s conjecture will be settled in the not too distant future. < 


EXAMPLE 8 There are many conjectures asserting that there are infinitely many primes of certain special 
forms. A conjecture of this sort is the conjecture that there are infinitely many primes of the 

lini » form n? + 1, where n is a positive integer. For example, 5 = 2? + 1,17 = 4° + 1,37=67 +1, 

and so on. The best result currently known is that there are infinitely many positive integers n 

such that n? + 1 is prime or the product of at most two primes (proved by Henryk Iwaniec in 1973 

using advanced techniques from analytic number theory, far beyond the scope of this book). <4 


EXAMPLE 9_ The Twin Prime Conjecture Twin primes are pairs of primes that differ by 2, such as 3 and 
5, 5 and 7, 11 and 13, 17 and 19, and 4967 and 4969. The twin prime conjecture asserts that 
Links > 


CHRISTIAN GOLDBACH (1690-1764) Christian Goldbach was born in Königsberg, Prussia, the city noted for its famous bridge 
problem (which will be studied in Section 10.5). He became professor of mathematics at the Academy in St. Petersburg in 1725. In 
1728 Goldbach went to Moscow to tutor the son of the Tsar. He entered the world of politics when, in 1742, he became a staff member 
in the Russian Ministry of Foreign Affairs. Goldbach is best known for his correspondence with eminent mathematicians, including 
Euler and Bernoulli, for his intriguing conjectures in number theory, and for several contributions to analysis. 
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Source: John D. & 
Catherine T. MacArthur 
Foundation 


there are infinitely many twin primes. The strongest result proved concerning twin primes is that 
there are infinitely many pairs p and p + 2, where p is prime and p + 2 is prime or the product 
of two primes (proved by J. R. Chen in 1966). 


The world’s record for twin primes, as of early 2018, consists of the numbers 
2,996, 863,034,895 - 2!:290,000 + 1, which have 388,342 decimal digits. 

Let P(n) be the statement that there are infinitely many pairs of primes that differ by exactly 
n. The twin prime conjecture is the statement that P(2) is true. Mathematicians working on the 
twin prime conjecture formulated a weaker conjecture, known as the bounded gap conjecture, 
which asserts that there is an integer N for which P(N) is true. The mathematical community 
was surprised when Yitang Zhang, a 50-year-old professor at the University of New Hampshire, 
who had not published a paper since 2001, proved the bounded gap conjecture in 2013. In 
particular, he showed that there is an integer N < 70,000,000 such that P(N) is true. A team of 
mathematicians, including Terrance Tao, lowered the Zhang’s bound by showing that there is 
an integer N < 246 for which P(N) is true. Furthermore, they showed that if a certain conjecture 
was true, it could be shown that N < 6 and that this is the best possible estimate that could be 
proved using the methods introduced by Zhang. < 


4.3.6 Greatest Common Divisors and Least Common Multiples 


The largest integer that divides both of two integers is called the greatest common divisor of 
these integers. 


Let a and b be integers, not both zero. The largest integer d such that d | a and d | bis called 
the greatest common divisor of a and b. The greatest common divisor of a and b is denoted 
by gcd(a, b). 


The greatest common divisor of two integers, not both zero, exists because the set of com- 
mon divisors of these integers is nonempty and finite. One way to find the greatest common 
divisor of two integers is to find all the positive common divisors of both integers and then take 
the largest divisor. This is done in Examples 10 and 11. Later, a more efficient method of finding 
greatest common divisors will be given. 


What is the greatest common divisor of 24 and 36? 


Solution: The positive common divisors of 24 and 36 are 1, 2, 3, 4, 6, and 12. Hence, 
gcd(24, 36) = 12. < 


YITANG ZHANG (BORN 1955) Yitang Zhang was born in Shanghai, China, in 1955. When he was ten 
years old, he learned about famous conjectures, including Fermat’s last theorem and the Goldbach conjecture. 
During the Cultural Revolution he spent ten years working in the fields instead of attending school. However, 
once this period was over, he was able to attend Peking University, receiving his bachelor’s and master’s degree 
in 1982 and 1984, respectively. He moved to the United States, attending Purdue University and completing the 
work for his Ph.D. in 1991. 

After receiving his Ph.D., Zhang could not find an academic position because of the poor job market and 
disagreements with his thesis advisor. Instead he did accounting work and delivered food for a Queens, New 
York restaurant; he later worked in Kentucky at Subway restaurants owned by a friend. He even lived in his car 
while looking for work, but was finally able to obtain an academic job as a lecturer at the University of New 
Hampshire. He held this position from 1999 until early 2014. From 2009 to 2013, he worked on the bounded 


gap conjecture seven days a week, about ten hours a day, until he made his key discovery. His success led the University of New 
Hampshire to promote him to full professorship. In 2015, however, he accepted the offer of a full professorship at the University of 
California, Santa Barbara. Zhang was a awarded a MacArthur Fellowship, also known as a Genius Award, in 2014. 
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What is the greatest common divisor of 17 and 22? 


Solution: The integers 17 and 22 have no positive common divisors other than 1, so that 
gcd(17, 22) = 1. < 


Because it is often important to specify that two integers have no common positive divisor 
other than 1, we have Definition 3. 


The integers a and b are relatively prime if their greatest common divisor is 1. 


By Example 11 it follows that the integers 17 and 22 are relatively prime, because 
gcd(17, 22) = 1. < 


Because we often need to specify that no two integers in a set of integers have a common 
positive divisor greater than 1, we make Definition 4. 


The integers a), dp, ..., a, are pairwise relatively prime if gcd(a; a,) = 1 whenever 1 <i < 
Sis 


Determine whether the integers 10, 17, and 21 are pairwise relatively prime and whether the 
integers 10, 19, and 24 are pairwise relatively prime. 


Solution: Because gcd(10, 17) = 1, gcd(10, 21) = 1, and gced(17, 21) = 1, we conclude that 10, 
17, and 21 are pairwise relatively prime. 

Because gcd(10, 24) =2> 1, we see that 10, 19, and 24 are not pairwise relatively 
prime. < 


Another way to find the greatest common divisor of two positive integers is to use the prime 
factorizations of these integers. Suppose that the prime factorizations of the positive integers a 
and b are 

a; a bi by b 

a =p P7 sepi b =p P; wepi, 
where each exponent is a nonnegative integer, and where all primes occurring in the prime 
factorization of either a or b are included in both factorizations, with zero exponents if necessary. 
Then gcd(a, b) is given by 


min(a,, b}) _min(a,, b2) ... pmin(a,, bp) 


gcd(a, b) = p, Py edie 


where min(x, y) represents the minimum of the two numbers x and y. To show that this formula 
for gcd(a, b) is valid, we must show that the integer on the right-hand side divides both a and b, 
and that no larger integer also does. This integer does divide both a and b, because the power of 
each prime in the factorization does not exceed the power of this prime in either the factorization 
of a or that of b. Further, no larger integer can divide both a and b, because the exponents of 
the primes in this factorization cannot be increased, and no other primes can be included. 
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Because the prime factorizations of 120 and 500 are 120 = 2? - 3 - 5 and 500 = 2? - 53, the great- 
est common divisor is 


sed( 120, 500) = 27) gmt, smn.) — 223051 = 20. <a 


Prime factorizations can also be used to find the least common multiple of two integers. 


The least common multiple of the positive integers a and b is the smallest positive integer that 
is divisible by both a and b. The least common multiple of a and b is denoted by Icm(a, b). 


The least common multiple exists because the set of integers divisible by both a and b is 
nonempty (because ab belongs to this set, for instance), and every nonempty set of positive 
integers has a least element (by the well-ordering property, which will be discussed in Section 
5.2). Suppose that the prime factorizations of a and b are as before. Then the least common 
multiple of a and b is given by 


max(a,,b,)__max(d, by) max(a,, b,) 
1 Pa =D, n $ 


Icm(a, b) = p 


where max(x, y) denotes the maximum of the two numbers x and y. This formula is valid because 


a common multiple of a and b has at least max(a;, b;) factors of p; in its prime factorization, and 


the least common multiple has no other prime factors besides those in a and b. 


What is the least common multiple of 23357? and 2433? 

Solution: We have 
lem(2°3°77 2433) = qimax(3, 4) 3max(5, 3) 7max(2, 0) = 243572., < 
Theorem 5 gives the relationship between the greatest common divisor and least common 


multiple of two integers. It can be proved using the formulae we have derived for these quantities. 
The proof of this theorem is left as Exercise 31. 


Let a and b be positive integers. Then 


ab = gcd(a, b) - lemla, b). 


4.3.7 The Euclidean Algorithm 


Computing the greatest common divisor of two integers directly from the prime factorizations 
of these integers is inefficient. The reason is that it is time-consuming to find prime factoriza- 
tions. We will give a more efficient method of finding the greatest common divisor, called the 
Euclidean algorithm. This algorithm has been known since ancient times. It is named after the 


LEMMA 1 
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ancient Greek mathematician Euclid, who included a description of this algorithm in his book 
The Elements. 

Before describing the Euclidean algorithm, we will show how it is used to find gcd(91, 287). 
First, divide 287, the larger of the two integers, by 91, the smaller, to obtain 


287 =91 -3 + 14. 


Any divisor of 91 and 287 must also be a divisor of 287 — 91 - 3 = 14. Also, any divisor of 91 
and 14 must also be a divisor of 287 = 91 - 3 + 14. Hence, the greatest common divisor of 91 
and 287 is the same as the greatest common divisor of 91 and 14. This means that the problem 
of finding gcd(91, 287) has been reduced to the problem of finding gcd(91, 14). 

Next, divide 91 by 14 to obtain 


91= 14-647. 


Because any common divisor of 91 and 14 also divides 91 — 14 - 6 = 7 and any common divisor 
of 14 and 7 divides 91, it follows that gcd(91, 14) = gced(14, 7). 
Continue by dividing 14 by 7, to obtain 


14=7.-2. 


Because 7 divides 14, it follows that gcd(14, 7) = 7. Furthermore, because gcd(287, 91) = 
gcd(91, 14) = gcd(14, 7) = 7, the original problem has been solved. 

We now describe how the Euclidean algorithm works in generality. We will use successive 
divisions to reduce the problem of finding the greatest common divisor of two positive integers 
to the same problem with smaller integers, until one of the integers is zero. 

The Euclidean algorithm is based on the following result about greatest common divisors 
and the division algorithm. 


Let a = bq + r, where a, b, q, and r are integers. Then gcd(a, b) = gcd(, r). 


Proof: If we can show that the common divisors of a and b are the same as the common divisors 
of b and r, we will have shown that gcd(a, b) = gcd(b, r), because both pairs must have the same 
greatest common divisor. 

So suppose that d divides both a and b. Then it follows that d also divides a — bq = r (from 
Theorem 1 of Section 4.1). Hence, any common divisor of a and b is also a common divisor 
of b and r. 

Likewise, suppose that d divides both b and r. Then d also divides bg + r = a. Hence, any 
common divisor of b and r is also a common divisor of a and b. 

Consequently, gcd(a, b) = gcd(b, r). < 


EUCLID (325 B.C.E.—265 B.C.E.) Euclid was the author of the most successful mathematics book ever writ- 
ten, The Elements, which appeared in over 1000 different editions from ancient to modern times. Little is known 
about Euclid’s life, other than that he taught at the famous academy at Alexandria in Egypt. Apparently, Euclid 
did not stress applications. When a student asked what he would get by learning geometry, Euclid explained 


| that knowledge was worth acquiring for its own sake and told his servant to give the student a coin “because he 


must make a profit from what he learns.” 
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Suppose that a and b are positive integers with a > b. Let rọ = a and r; = b. When we 
successively apply the division algorithm, we obtain 


% =", +r OS <r, 
Ny =n ths OST <T, 


Ta-2 = fn-14n-1 TT n 0 S Tp < Ta- 
Tn-1 = niw 
Eventually a remainder of zero occurs in this sequence of successive divisions, because the 
sequence of remainders a = rọ > r} > r, > +++ = 0 cannot contain more than a terms. Further- 
more, it follows from Lemma | that 
gcd(a, b) = gcd(ro, rı) = ged(r, r2) T ged(r,,_», Fi) 
= gced(r,,_),7,) = gced(r,, 0) = r, 


Hence, the greatest common divisor is the last nonzero remainder in the sequence of divisions. 


EXAMPLE 16 Find the greatest common divisor of 414 and 662 using the Euclidean algorithm. 


Solution: Successive uses of the division algorithm give: 


662 = 414- 1+ 248 
414 = 248 - 1+ 166 
248 = 166-1+ 82 
166 = 82-242 
82=2-41. 


Hence, gcd(414, 662) = 2, because 2 is the last nonzero remainder. 
We can summarize these steps in tabular form. 


3o fiaa fj Yj 
662 414 1 248 
414 248 1 166 
248 166 1 82 
166 82 2 2 
82 2 41 0 < 


J 
0 
1 
2 
3 
4 


The Euclidean algorithm is expressed in pseudocode in Algorithm 1. 


ALGORITHM 1 The Euclidean Algorithm. 


procedure gcd(a, b: positive integers) 


x:=a 
y:=b 
while y 4 0 
r := x mod y 
x:=y 
y:=r 


return x{gcd(a, b) is x} 
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In Algorithm 1, the initial values of x and y are a and b, respectively. At each stage of the 
procedure, x is replaced by y, and y is replaced by x mod y, which is the remainder when x is 
divided by y. This process is repeated as long as y # 0. The algorithm terminates when y = 0, 
and the value of x at that point, the last nonzero remainder in the procedure, is the greatest 
common divisor of a and b. 

We will study the time complexity of the Euclidean algorithm in Section 5.3, where we 
will show that the number of divisions required to find the greatest common divisor of a and b, 
where a > b, is O(log b). 


4.3.8 gcds as Linear Combinations 


An important result we will use throughout the remainder of this section is that the greatest 
common divisor of two integers a and b can be expressed in the form 


sa + tb, 


where s and ¢ are integers. In other words, gcd(a, b) can be expressed as a linear combination 
with integer coefficients of a and b. For example, gcd(6, 14) = 2, and 2 = (—2)-6+1- 14. We 
state this fact as Theorem 6. 


BEZOUT’S THEOREM _ If aand bare positive integers, then there exist integers s and t 
such that gcd(a, b) = sa + tb. 


If a and b are positive integers, then integers s and ¢ such that gcd(a, b) = sa + tb are called 
Bézout coefficients of a and b (after Etienne Bézout, a French mathematician of the eighteenth 
century). Also, the equation gcd(a, b) = sa + tb is called Bézout’s identity. 


We will not give a formal proof of Theorem 6 here (see Exercise 36 in Section 5.2 and [Ro10] 
for proofs). We will present two different methods that can be used to find a linear combination 
of two integers equal to their greatest common divisor. (In this section, we will assume that a 
linear combination has integer coefficients.) 

The first method proceeds by working backward through the divisions of the Euclidean 
algorithm, so this method requires a forward pass and a backward pass through the steps of 
the Euclidean algorithm. We will illustrate how this method works with an example. The main 


ETIENNE BEZOUT (1730-1783) Bézout was born in Nemours, France, where his father was a magistrate. 
Reading the writings of the great mathematician Leonhard Euler enticed him to become a mathematician. In 
1758 he was appointed to a position at the Académie des Sciences in Paris; in 1763 he was appointed examiner of 
the Gardes de la Marine, where he was assigned the task of writing mathematics textbooks. This assignment led 
to a four-volume textbook completed in 1767. Bézout is well known for his six-volume comprehensive textbook 
on mathematics. His textbooks were extremely popular and were studied by many generations of students hoping 
to enter the Ecole Polytechnique, the highly regarded engineering and science school. His books were translated 
into English and used in North America, including at Harvard. 

His most important original work was published in 1779 in the book Théorie générale des équations 
algébriques, where he introduced important methods for solving simultaneous polynomial equations in many 
unknowns. The most well-known result in this book is now called Bézout’s theorem, which in its general 


form tells us that the number of common points on two plane algebraic curves equals the product of the degrees of these curves. 
Bézout is also credited with inventing the determinant (which was called the Bézoutian by the noted English mathematician James 
Joseph Sylvester). He was considered to be a kind person with a warm heart, although he had a reserved and somber personality. 
He was happily married and a father. 
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EXAMPLE 17 


advantage of the second method, known as the extended Euclidean algorithm, is that it uses 
one pass through the steps of the Euclidean algorithm to find Bézout coefficients of a and b, 
unlike the first method, which uses two passes. To run this extended Euclidean algorithm we 
set sọ = 1, sı = 0, fọ = 0, and ft, = 1 and let 


Sj = Sj-2 — Gj-18;-) and t; = t2 — Qt 


forj = 2, 3, ..., n, where the q; are the quotients in the divisions used when the Euclidean algo- 
rithm finds gcd(a, b), as shown in the text. We can prove by strong induction (see Exercise 44 
in Section 5.2, or see [Ro10]) that gcd(a, b) = s,a+t,b. 


Express gcd(252, 198) = 18 as a linear combination of 252 and 198 by working backwards 
through the steps of the Euclidean algorithm. 


Solution: To show that gcd(252, 198) = 18, the Euclidean algorithm uses these divisions: 
252 = 198 . 1 + 54 


198 = 54 -3 + 36 
54 =36.1+18 
36 = 18-240. 


We summarize these steps in tabular form: 


Using the next-to-last division (the third division), we can express gcd(252, 198) = 18 as a linear 
combination of 54 and 36. We find that 


18 = 54 — 1 - 36. 
The second division tells us that 
36 = 198 — 3 - 54. 


Substituting this expression for 36 into the previous equation, we can express 18 as a linear 
combination of 54 and 198. We have 


18 = 54 — 1 . 36 = 54 — 1 . (198 — 3 . 54) = 4 . 54 — 1 - 198. 
The first division tells us that 
54 = 252 — 1 . 198. 


Substituting this expression for 54 into the previous equation, we can express 18 as a linear 
combination of 252 and 198. We conclude that 


18 = 4 - (252 — 1 - 198) — 1 - 198 = 4 - 252 — 5 - 198, 


completing the solution. < 


The next example shows how to solve the same problem posed in the previous example 
using the extended Euclidean algorithm. 


EXAMPLE 18 
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Express gcd(252, 198) = 18 as a linear combination of 252 and 198 using the extended Eu- 
clidean algorithm. 


Solution: Example 17 displays the steps the Euclidean algorithm uses to find gcd(252, 198) = 
18. The quotients are q; = 1, q) = 3, q3 = 1, and q4 = 2. The desired Bézout coefficients are the 
values of s, and t, generated by the extended Euclidean algorithm, where sọ = 1, s} = 0, tọ = 0, 
and ¢, = 1, and 


Sj = Sj-2 T GJ-1Sj-1 and = G2 7 Gh 
for j = 2, 3, 4. We find that 


S2 = So — s14, =1-0.-1=1, h =t -tq =0-1.1=-l1, 


Because s4 = 4 and t4 = —5, we see that 18 = gcd(252, 198) = 4-252 — 5 . 198. 
We summarize the steps of the extended Euclidean algorithm in a table: 


J r Vist Gar P Sj t; 
0 |252 198 1 54 1 (0) 
1 | 198 54 3 36 0 1 
2 54 36 1 18 1 -l 
3 36 18 2 0 -3 4 
4 4 -5 < 


We will use Theorem 6 to develop several useful results. One of our goals will be to prove 
the part of the fundamental theorem of arithmetic asserting that a positive integer has at most 
one prime factorization. We will show that if a positive integer has a factorization into primes, 
where the primes are written in nondecreasing order, then this factorization is unique. 

First, we need to develop some results about divisibility. 


If a, b, and c are positive integers such that gcd(a, b) = 1 anda | bc, thena | c. 


Proof: Because gcd(a, b) = 1, by Bézout’s theorem there are integers s and t such that 


sa+tb=1. 


Multiplying both sides of this equation by c, we obtain 


sac + tbc =c. 


We can now use Theorem 1 of Section 4.1 to show that a | c. By part (ii ) of that theorem, a | tbc. 
Because a | sac and a | tbc, by part (i) of that theorem, we conclude that a divides sac + tbc. 
Because sac + thc = c, we conclude that a | c, completing the proof. < 


We will use the following generalization of Lemma 2 in the proof of uniqueness of prime 
factorizations. (The proof of Lemma 3 is left as Exercise 64 in Section 5.1, because it can be 
most easily carried out using the method of mathematical induction, covered in that section.) 


If p is a prime and p | aja, =+ a„, where each a; is an integer, then p | a; for some i. 
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We can now show that a factorization of an integer into primes is unique. That is, we will 
show that every integer can be written as the product of primes in nondecreasing order in at 
most one way. This is part of the fundamental theorem of arithmetic. We will prove the other 
part, that every integer has a factorization into primes, in Section 5.2. 


Proof (of the uniqueness of the prime factorization of a positive integer): We will use a proof 
by contradiction. Suppose that the positive integer n can be written as the product of primes in 
two different ways, say, n = pıP2 ++" Ps and n = q,q> + g,, where each p; and q; is prime such 
that p; Sp, < + < p, and q) <q, S +" Sq. 

When we remove all common primes from the two factorizations, we have 


Pi,Pi, Pu = Ti, U, i 


where no prime occurs on both sides of this equation and u and v are positive integers. By 
Lemma 3 it follows that p; divides g, for some k. Because no prime divides another prime, 
this is impossible. Consequently, there can be at most one factorization of n into primes in 
nondecreasing order. < 


Lemma 2 can also be used to prove a result about dividing both sides of a congruence by 
the same integer. We have shown (Theorem 5 in Section 4.1) that we can multiply both sides of 
a congruence by the same integer. However, dividing both sides of a congruence by an integer 
does not always produce a valid congruence, as Example 19 shows. 


The congruence 14 = 8 (mod 6) holds, but both sides of this congruence cannot be divided by 2 
to produce a valid congruence because 14/2 = 7 and 8/2 = 4, but 7 £ 4 (mod 6). < 


Although we cannot divide both sides of a congruence by any integer to produce a valid 
congruence, we can if this integer is relatively prime to the modulus. Theorem 7 establishes this 
important fact. We use Lemma 2 in the proof. 


THEOREM 7 Let m be a positive integer and let a, b, and c be integers. If ac = bc (mod m) and 
gcd(c,m) = 1, then a = b (mod m). 
Proof: Because ac = bc (mod m), m | ac — bc = c(a — b). By Lemma 2, because ged(c, m) =1, 
it follows that m | a — b. We conclude that a = b (mod m). < 
Exercises 
1. Determine whether each of these integers is prime. 4. Find the prime factorization of each of these integers. 

a) 21 b) 29 a) 39 b) 81 c) 101 

9 71 d) 97 d) 143 e) 289 f) 899 

e) 111 f) 143 5. Find the prime factorization of 10!. 


2. Determine whether each of these integers is prime. 


*6. How many zeros are there at the end of 100!? 
7. Express in pseudocode the trial division algorithm for de- 


a) 19 b) 27 we : age 
©) 93 d) 101 termining whether an integer is prime. 
8. Express in pseudocode the algorithm described in the text 
e) 107 f) 113 : : BA : 
‘ . a . for finding the prime factorization of an integer. 
3. Find the prime factorization of each of these integers. 9. Show that a” + 1 is composite if a and m are integers 
a) 88 b) 126 c) 729 greater than 1 and m is odd. [Hint: Show that x + 1 is a 
d) 1001 e) 1111 f) 909,090 factor of the polynomial x” + 1 if m is odd.] 


10 


*11 


12 


*13 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


. Show that if 2” +1 is an odd prime, then m = 2” 
for some nonnegative integer n. [Hint: First show 
that the polynomial identity x” + 1 = (x* + Dako) — 
XD 44. — xk 41) holds, where m=kt and t 
is odd.] 

. Show that log, 3 is an irrational number. Recall that an ir- 
rational number is a real number x that cannot be written 
as the ratio of two integers. 


. Prove that for every positive integer n, there are n con- 
secutive composite integers. [Hint: Consider the n con- 
secutive integers starting with (n + 1)! + 2.] 

. Prove or disprove that there are three consecutive odd 

positive integers that are primes, that is, odd primes of 

the form p, p + 2, and p + 4. 

Which positive integers less than 12 are relatively prime 

to 12? 


Which positive integers less than 30 are relatively prime 
to 30? 

Determine whether the integers in each of these sets are 
pairwise relatively prime. 
a) 21, 34,55 

c) 25, 41, 49, 64 


b) 14, 17, 85 
d) 17, 18, 19, 23 


Determine whether the integers in each of these sets are 
pairwise relatively prime. 

a) 11, 15,19 b) 14, 15, 21 

¢) 12,.17,31,37 d) 7, 8,9, 11 


We call a positive integer perfect if it equals the sum of 

its positive divisors other than itself. 

a) Show that 6 and 28 are perfect. 

b) Show that 2?-'(2? — 1) is a perfect number when 
2? — 1 is prime. 

Show that if 2” — 1 is prime, then n is prime. [Hint: Use 

the identity 2% — 1 = (24 — 1) - (247D + 29-2) 4 + 

2% + 1).] 

Determine whether each of these integers is prime, veri- 

fying some of Mersenne’s claims. 

a) 27-1 b) 2-1 

c) 2-1 d) 23-1 


The value of the Euler -function at the positive integer n is 
defined to be the number of positive integers less than or equal 
to n that are relatively prime to n. For instance, (6) = 2 be- 
cause of the positive integers less or equal to 6, only 1 and 5 
are relatively prime to 6. [Note: ¢ is the Greek letter phi.] 


21 


22. 
23. 


24. 


. Find these values of the Euler @-function. 

a) p(4) b) (10) c) (13) 

Show that n is prime if and only if ġ(n) =n- 1. 

What is the value of ¢(p") when p is prime and k is a 
positive integer? 

What are the greatest common divisors of these pairs of 
integers? 

a) 22.33.55, 25.33.52 

b) 2-3-5-7-11-13,2! 3°. 11-17“ 

ce) 17,17!" d) 27-7, 53-13 

e) 0,5 F) 2-3-5-7,2-3-5-7 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35; 


*36. 


S * 37, 


38. 


39. 
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What are the greatest common divisors of these pairs of 
integers? 

a) 37.59.73, 21! 35.59 

b) ae a 7 

c) 2371, 2317 

d) 41-43-53, 41-43-53 

e) 313 517 212 i 721 

f) 1111,0 

What is the least common multiple of each pair in Exer- 
cise 24? 

What is the least common multiple of each pair in Exer- 
cise 25? 

Find gced(1000, 625) and Icm(1000, 625) and verify that 
gcd(1000, 625) - lem(1000, 625) = 1000 - 625. 

Find gcd(92928, 123552) and Icm(92928, 123552), and 
verify that gcd(92928, 123552) - Icm(92928, 123552) = 
92928 - 123552. [Hint: First find the prime factorizations 
of 92928 and 123552.] 

If the product of two integers is 2738577! and their great- 
est common divisor is 23345, what is their least common 
multiple? 

Show that if a and b are positive integers, then ab = 
gcd(a, b) - lcm(a, b). [Hint: Use the prime factorizations 
of a and b and the formulae for gcd(a, b) and Icm(a, b) in 
terms of these factorizations. | 

Use the Euclidean algorithm to find 

a) gcd(1, 5). b) gcd(100, 101). 

c) gced(123, 277). d) gcd(1529, 14039). 

e) gced(1529, 14038). f) gced(11111, 111111). 
Use the Euclidean algorithm to find 

a) gcd(12, 18). b) gcd(111, 201). 

c) gced(1001, 1331). d) gcd(12345, 54321). 

e) gcd(1000, 5040). f) gcd(9888, 6060). 

How many divisions are required to find gcd(21, 34) us- 
ing the Euclidean algorithm? 

How many divisions are required to find gcd(34, 55) us- 
ing the Euclidean algorithm? 

Show that if a and b are both positive integers, then 
(2% — 1) mod (2° — 1) = 24 med b _ 1, 

Use Exercise 36 to show that if a and b are pos- 
itive integers, then gcd(2¢— 1,2? — 1) = 284 b) _ 1, 
[Hint: Show that the remainders obtained when the Eu- 
clidean algorithm is used to compute ged(2¢ — 1, 2 — 1) 
are of the form 2” — 1, where r is a remainder arising 
when the Euclidean algorithm is used to find gcd(a, b).] 
Use Exercise 37 to show that the integers 235 — 1,234 — 1, 
233 — 1, 23! 1, 29 — 1, and 273 — 1 are pairwise rela- 
tively prime. 

Using the method followed in Example 17, express the 
greatest common divisor of each of these pairs of inte- 
gers as a linear combination of these integers. 


a) 10,11 b) 21, 44 c) 36, 48 
d) 34,55 e) 117,213 f) 0,223 
g) 123, 2347 h) 3454,4666 i) 9999, 11111 


290 4/ Number Theory and Cryptography 


40. 


41. 


42. 


43. 


44. 


45. 


46. 


47. 


48. 


49. 


Using the method followed in Example 17, express the 
greatest common divisor of each of these pairs of inte- 
gers as a linear combination of these integers. 

a) 9, 11 b) 33, 44 c) 35,78 

d) 21,55 e) 101, 203 f) 124, 323 

g) 2002,2339 h) 3457,4669 i) 10001, 13422 
Use the extended Euclidean algorithm to express 
gcd(26, 91) as a linear combination of 26 and 91. 

Use the extended Euclidean algorithm to express 
gcd(252, 356) as a linear combination of 252 and 356. 
Use the extended Euclidean algorithm to express 
gcd(144, 89) as a linear combination of 144 and 89. 

Use the extended Euclidean algorithm to express 
gcd(1001, 100001) as a linear combination of 1001 and 
100001. 

Describe the extended Euclidean algorithm using pseu- 
docode. 

Find the smallest positive integer with exactly n different 
positive factors when n is 
a) 3. b) 4. 

d) 6. e) 10. 
Can you find a formula or rule for the nth term of a se- 
quence related to the prime numbers or prime factoriza- 
tions so that the initial terms of the sequence have these 
values? 


c) 5. 


? 


11, 
f) 1, 2, 6, 30, 210, 2310, 30030, 
223092870, ... 
Can you find a formula or rule for the nth term of a se- 
quence related to the prime numbers or prime factoriza- 
tions so that the initial terms of the sequence have these 
values? 


3, 13, . 
0510, 9699690, 


a) 2,23337; T, 11, 11,1111; 13,13, 

b) 0, 1, 2, 2, 3, 3, 4, 4, 4, 4, 5, 5, 6, 6, ... 

c) 1,0,0, 1,0, 1,0, 1, 1, 1,0, 1,0, 1, 

d) 1, —1, —1, 0, -1, T 1,-1,0,-1,1,1,... 
e) 1111101; 110,101, 


0,0,. 
f) 4,9, 25, 49, 121, 169, 289, 361, 529, 841, 961, 1369, . 


Prove that the product of any three consecutive ana 
is divisible by 6. 


Solving Congruences 


50. 


#51. 


52. 


53. 


54. 


55. 


* 56. 


*57, 


Show that if a, b, and m are integers such that m > 2 and 
a = b (mod m), then gcd(a, m) = gced(b, m). 


Prove or disprove that n? — 79n + 1601 is prime when- 
ever n is a positive integer. 


Prove or disprove that p;p +P, + 1 is prime for every 
positive integer n, where p}, Pz ...,p,, are the n smallest 
prime numbers. 


Show that there is a composite integer in every arithmetic 
progression ak + b, k = 1, 2, ..., where a and b are posi- 
tive integers. 


Adapt the proof in the text that there are infinitely many 
primes to prove that there are infinitely many primes 
of the form 3k+2, where k is a nonnegative inte- 
ger. [Hint: Suppose that there are only finitely many 
such primes qj, qz ---> q,» and consider the number 


34192 dn 1] 


Adapt the proof in the text that there are infinitely many 
primes to prove that there are infinitely many primes 
of the form 4k+3, where k is a nonnegative inte- 
ger. [Hint: Suppose that there are only finitely many 
such primes qı, q2, ---, qn, and consider the number 
44,92 dy — 1] 


Prove that the set of positive rational numbers is count- 
able by setting up a function that assigns to a ratio- 
nal number p/q with gcd(p, q) = 1 the base 11 number 
formed by the decimal representation of p followed by 
the base 11 digit A, which corresponds to the decimal 
number 10, followed by the decimal representation of q. 


Prove that the set of positive rational numbers is count- 
able by showing that the function K is a one-to- 
one correspondence between the set of positive ra- 


tional numbers and the set of positive integers if 
2a; 2a, 2a, 2b}—1 2b,—1 2b,-1 
K(m/n) = pi Pa PL! Ge 
where gcd(m, n) = 1 and the prime; power tactorizalions 
b 

++ pes ‘andn = q? q? 


t 


of m and n are m = Pi po 


4.4.1 Introduction 


Solving linear congruences, which have the form ax = b (mod m), is an essential task in the 
study of number theory and its applications, just as solving linear equations plays an important 
role in calculus and linear algebra. To solve linear congruences, we employ inverses modulo 
m. We explain how to work backwards through the steps of the Euclidean algorithm to find 
inverses modulo m. Once we have found an inverse of a modulo m, we solve the congruence 
ax = b (mod m) by multiplying both sides of the congruence by this inverse. 
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Simultaneous systems of linear congruence have been studied since ancient times. For ex- 
ample, the Chinese mathematician Sun-Tsu studied them in the first century. We will show how 
to solve systems of linear congruences modulo pairwise relatively prime moduli. The result we 
will prove is called the Chinese remainder theorem, and our proof will give a method to find all 
solutions of such systems of congruences. We will also show how to use the Chinese remainder 
theorem as a basis for performing arithmetic with large integers. 

We will introduce a useful result of Fermat, known as Fermat’s little theorem, which states 
that if p is prime and p does not divide a, then a’~! = 1 (mod p). We will examine the converse 
of this statement, which will lead us to the concept of a pseudoprime. A pseudoprime m to 
the base a is a composite integer m that masquerades as a prime by satisfying the congruence 
a"! = 1 (mod m). We will also give an example of a Carmichael number, which is a composite 
integer that is a pseudoprime to all bases a relatively prime to it. 

We also introduce the notion of discrete logarithms, which are analogous to ordinary log- 
arithms. To define discrete logarithms we must first define primitive roots. A primitive root of 
a prime p is an integer r such that every integer not divisible by p is congruent to a power of r 
modulo p. If r is a primitive root of p and r° = a (mod p), then e is the discrete logarithm of a 
modulo p to the base r. Finding discrete logarithms turns out to be an extremely difficult prob- 
lem in general. The difficulty of this problem is the basis for the security of many cryptographic 
systems. 


4.4.2 Linear Congruences 


A congruence of the form 


ax = b (mod m), 


where m is a positive integer, a and b are integers, and x is a variable, is called a linear congru- 
ence. Such congruences arise throughout number theory and its applications. 

How can we solve the linear congruence ax = b (mod m), that is, how can we find all 
integers x that satisfy this congruence? One method that we will describe uses an integer a 
such that aa = 1 (mod m), if such an integer exists. Such an integer a is said to be an inverse 
of a modulo m. Theorem 1 guarantees that an inverse of a modulo m exists whenever a and m 
are relatively prime. 


If a and m are relatively prime integers and m > 1, then an inverse of a modulo m exists. 
Furthermore, this inverse is unique modulo m. (That is, there is a unique positive integer a 
less than m that is an inverse of a modulo m and every other inverse of a modulo m is con- 
gruent to a modulo m.) 


Proof: By Theorem 6 of Section 4.3, because gcd(a, m) = 1, there are integers s and t such that 


sattm=1. 


This implies that 


sa + tm = 1 (mod m). 


Because tm = 0 (mod m), it follows that 
sa = 1 (mod m). 


Consequently, s is an inverse of a modulo m. That this inverse is unique modulo m is left as 
Exercise 7. < 
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EXAMPLE 1 


EXAMPLE 2 


Using inspection to find an inverse of a modulo m is easy when m is small. To find this 
inverse, we look for a multiple of a that exceeds a multiple of m by 1. For example, to find an 
inverse of 3 modulo 7, we can find j - 3 for j = 1, 2, ..., 6, stopping when we find a multiple of 
3 that is one more than a multiple of 7. We can speed this approach up if we note that 2-3 = 
—1 (mod 7). This means that (—2) - 3 = 1 (mod 7). Hence, 5 - 3 = 1 (mod 7), so 5 is an inverse 
of 3 modulo 7. 

We can design a more efficient algorithm than brute force to find an inverse of a modulo m 
when gcd(a, m) = 1 using the steps of the Euclidean algorithm. By reversing these steps as 
in Example 17 of Section 4.3, we can find a linear combination sa + tm = 1, where s and t 
are integers. Reducing both sides of this equation modulo m tells us that s is an inverse of 
a modulo m. We illustrate this procedure in Example 1. 


Find an inverse of 3 modulo 7 by first finding Bézout coefficients of 3 and 7. (Note that we have 
already shown that 5 is an inverse of 3 modulo 7 by inspection.) 


Solution: Because gcd(3, 7) = 1, Theorem 1 tells us that an inverse of 3 modulo 7 exists. The 
Euclidean algorithm ends quickly when used to find the greatest common divisor of 3 and 7: 


7=2-34+1. 
From this equation we see that 
—2:-34+1-7=1. 


This shows that —2 and 1 are Bézout coefficients of 3 and 7. We see that —2 is an inverse of 3 
modulo 7. Note that every integer congruent to —2 modulo 7 is also an inverse of 3, such as 5, 
—9, 12, and so on. < 


Find an inverse of 101 modulo 4620. 


Solution: For completeness, we present all steps used to compute an inverse of 101 modulo 4620. 
(Only the last step goes beyond methods developed in Section 4.3 and illustrated in Example 17 
in that section.) First, we use the Euclidean algorithm to show that gcd(101, 4620) = 1. Then 
we will reverse the steps to find Bézout coefficients a and b such that 101a + 4620b = 1. It will 
then follow that a is an inverse of 101 modulo 4620. The steps used by the Euclidean algorithm 
to find gcd(101, 4620) are 


4620 = 45 - 101 +75 
101 =1-75+ 26 


75 = 2-264 23 
26 = 1-23+3 
23=7-34+2 
3=1-2+1 
2=2-1. 


Because the last nonzero remainder is 1, we know that gcd(101, 4620) = 1. We can now find 
the Bézout coefficients for 101 and 4620 by working backwards through these steps, expressing 


EXAMPLE 3 


Links » 


EXAMPLE 4 
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gcd(101, 4620) = 1 in terms of each successive pair of remainders. In each step we elimi- 
nate the remainder by expressing it as a linear combination of the divisor and the dividend. 
We obtain 


1=3-1-2 
=3-1-(3-7-3)=-1-234+8-3 
=—-1-234+8-(26-—1-23)=8-26-—9- 23 
=8-26-—9-(75—2- 26) =—9-75 + 26- 26 
= —9 .75 4+ 26-(101 — 1-75) = 26-101 — 35-75 
= 26-101 — 35 - (4620 — 45 - 101) = —35 - 4620 + 1601 - 101. 


That —35 - 4620 + 1601 - 101 = 1 tells us that —35 and 1601 are Bézout coefficients of 4620 
and 101, and 1601 is an inverse of 101 modulo 4620. < 


Once we have an inverse a of a modulo m, we can solve the congruence ax = b (mod m) 
by multiplying both sides of the linear congruence by a, as Example 3 illustrates. 


What are the solutions of the linear congruence 3x = 4 (mod 7)? 


Solution: By Example 1 we know that —2 is an inverse of 3 modulo 7. Multiplying both sides 
of the congruence by —2 shows that 


—2 -3x = -2-4(mod 7). 


Because —6 = | (mod 7) and —8 = 6 (mod 7), it follows that if x is a solution, then x = —8 = 
6 (mod 7). 

We need to determine whether every x with x = 6 (mod 7) is a solution. Assume that x = 
6(mod 7). Then, by Theorem 5 of Section 4.1, it follows that 


3x =3-6=18 =4(mod 7), 


which shows that all such x satisfy the congruence. We conclude that the solutions to the 
congruence are the integers x such that x =6(mod 7), namely, 6, 13,20,... and —1, —8, 
—15,.... < 


4.4.3 The Chinese Remainder Theorem 


Systems of linear congruences arise in many contexts. For example, as we will see later, they are 
the basis for a method that can be used to perform arithmetic with large integers. Such systems 
can even be found as word puzzles in the writings of ancient Chinese and Hindu mathematicians, 
such as that given in Example 4. 


In the first century, the Chinese mathematician Sun-Tsu asked: 


There are certain things whose number is unknown. When divided by 3, the remainder 
is 2; when divided by 5, the remainder is 3; and when divided by 7, the remainder is 2. 
What will be the number of things? 
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THEOREM 2 


This puzzle can be translated into the following question: What are the solutions of the 
systems of congruences 


x = 2 (mod 3), 
x = 3 (mod 5), 
x = 2 (mod 7)? 
We will solve this system, and with it Sun-Tsu’s puzzle, later in this section. < 


The Chinese remainder theorem, named after the Chinese heritage of problems involving 
systems of linear congruences, states that when the moduli of a system of linear congruences 
are pairwise relatively prime, there is a unique solution of the system modulo the product of the 
moduli. 


THE CHINESE REMAINDER THEOREM Let m,, m, ..., m, be pairwise relatively 
prime positive integers greater than one and a, a, ..., a, arbitrary integers. Then the system 


x = a, (mod m,), 
x = a, (mod m,), 


x =a, (mod m,) 


has a unique solution modulo m=m,m,---m,. (That is, there is a solution x with 
0 < x < m, and all other solutions are congruent modulo m to this solution.) 


Proof: To establish this theorem, we need to show that a solution exists and that it is unique 
modulo m. We will show that a solution exists by describing a way to construct this solution; 
showing that the solution is unique modulo m is Exercise 30. 

To construct a simultaneous solution, first let 


M, = m/m; 


fork = 1, 2, ..., n. That is, M, is the product of the moduli except for m,. Because m; and m, have 
no common factors greater than 1 when i +£ k, it follows that gcd(m,, M,) = 1. Consequently, 
by Theorem 1, we know that there is an integer y,, an inverse of M, modulo m,, such that 


My; = 1 (mod m;,). 
To construct a simultaneous solution, form the sum 
x =a; My; +a My + ++ + ap, Mp Yn- 


We will now show that x is a simultaneous solution. First, note that because M;=0 (mod m,) 
whenever j + k, all terms except the kth term in this sum are congruent to 0 modulo m,. Because 
M,); = 1 (mod m,) we see that 


x =a,M,y, = a, (mod m,), 


for k = 1, 2, ..., n. We have shown that x is a simultaneous solution to the n congruences. < 


EXAMPLE 5 


EXAMPLE 6 
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Example 5 illustrates how to use the construction given in our proof of the Chinese remain- 
der theorem to solve a system of congruences. We will solve the system given in Example 4, 
which arises in Sun-Tsu’s puzzle. 


To solve the system of congruences in Example 4, first let m = 3-5-7 = 105, M, = m/3 = 
35, M, = m/5 = 21, and M; = m/7 = 15. We see that 2 is an inverse of M, = 35 modulo 3, 
because 35-2 =2-2=1 (mod 3); 1 is an inverse of M, = 21 modulo 5, because 21 = 
1 (mod 5); and 1 is an inverse of M} = 15 (mod 7), because 15 = 1 (mod 7). The solutions 
to this system are those x such that 


x = a,M,y, + Moy. + a3M3y3 = 2-35-24+3-21-14+2-15-1 
= 233 = 23 (mod 105). 


It follows that 23 is the smallest positive integer that is a simultaneous solution. We conclude that 
23 is the smallest positive integer that leaves a remainder of 2 when divided by 3, a remainder 
of 3 when divided by 5, and a remainder of 2 when divided by 7. < 


Although the construction in Theorem 2 provides a general method for solving systems 
of linear congruences with pairwise relatively prime moduli, it can be easier to solve a system 
using a different method. Example 6 illustrates the use of a method known as back substitution. 


Use the method of back substitution to find all integers x such that x=1 (mod 5), 
x =2 (mod 6), and x = 3 (mod 7). 


Solution: By Theorem 4 in Section 4.1, the first congruence can be rewritten as an equality, 
x = 5t + 1, where t is an integer. Substituting this expression for x into the second congruence 
tells us that 


5t+1=2(mod 6), 


which can be solved to show that t = 5 (mod 6) (as the reader should verify). Using Theorem 4 
in Section 4.1 again, we see that t = 6u + 5, where u is an integer. Substituting this expression 
for t back into the equation x = 5t + 1 tells us that x = 5(6u +5) + 1 = 30u + 26. We insert this 
into the third equation to obtain 


30u + 26 = 3 (mod 7). 


Solving this congruence tells us that u = 6 (mod 7) (as the reader should verify). Hence, Theo- 
rem 4 in Section 4.1 tells us that u = 7v + 6, where v is an integer. Substituting this expression 
for u into the equation x = 30u + 26 tells us thatx = 30(7v + 6) + 26 = 210u + 206. Translating 
this back into a congruence, we find the solution to the simultaneous congruences, 


x = 206 (mod 210). < 


4.4.4 Computer Arithmetic with Large Integers 


Suppose that m;, m», ...,m,, ate pairwise relatively prime moduli and let m be their product. 
By the Chinese remainder theorem, we can show (see Exercise 28) that an integer a with 0 < 
a < m can be uniquely represented by the n-tuple consisting of its remainders upon division by 
m, i= 1,2,...,n. That is, we can uniquely represent a by 


(a mod m, a mod m,, ...,a mod m,). 
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EXAMPLE 7 


EXAMPLE 8 


What are the pairs used to represent the nonnegative integers less than 12 when they are rep- 
resented by the ordered pair where the first component is the remainder of the integer upon 
division by 3 and the second component is the remainder of the integer upon division by 4? 


Solution: We have the following representations, obtained by finding the remainder of each 
integer when it is divided by 3 and by 4: 


0=(0,0) 4=(1,0) 8=(2,0) 
1=(1,1) 5=21 9=(0,1) 

2=(2,2) 6=(0,2) 10=(1,2) 

3=(0,3) 7=(1,3) 11 = (2,3). < 


To perform arithmetic with large integers, we select moduli m,, m, ..., m,, where each m; 
is an integer greater than 2, gcd(m;, m;) = 1 whenever i # j, and m = m,m, --- m, is greater than 
the results of the arithmetic operations we want to carry out. 

Once we have selected our moduli, we carry out arithmetic operations with large inte- 
gers by performing componentwise operations on the n-tuples representing these integers using 
their remainders upon division by m, i = 1, 2, ...,n. Once we have computed the value of each 
component in the result, we recover its value by solving a system of n congruences modulo 
m, i= 1,2,...,n. This method of performing arithmetic with large integers has several valu- 
able features. First, it can be used to perform arithmetic with integers larger than can ordinarily 
be carried out on a computer. Second, computations with respect to the different moduli can be 
done in parallel, speeding up the arithmetic. 


Suppose that performing arithmetic with integers less than 100 on a certain processor is much 
quicker than doing arithmetic with larger integers. We can restrict almost all our computations to 
integers less than 100 if we represent integers using their remainders modulo pairwise relatively 
prime integers less than 100. For example, we can use the moduli of 99, 98, 97, and 95. (These 
integers are relatively prime pairwise, because no two have a common factor greater than 1.) 

By the Chinese remainder theorem, every nonnegative integer less than 99 - 98 - 97 . 95 = 
89,403,930 can be represented uniquely by its remainders when divided by these four mod- 
uli. For example, we represent 123,684 as (33, 8, 9, 89), because 123,684 mod 99 = 33; 
123,684 mod 98 = 8; 123,684 mod 97 = 9; and 123,684 mod 95 = 89. Similarly, we represent 
413,456 as (32, 92, 42, 16). 

To find the sum of 123,684 and 413,456, we work with these 4-tuples instead of these two 
integers directly. We add the 4-tuples componentwise and reduce each component with respect 
to the appropriate modulus. This yields 


(33, 8, 9, 89) + (32, 92, 42, 16) 
= (65 mod 99, 100 mod 98, 51 mod 97, 105 mod 95) 
= (65, 2, 51, 10). 


To find the sum, that is, the integer represented by (65, 2, 51, 10), we need to solve the 
system of congruences 


x = 65 (mod 99), 
x= 2(mod 98), 
x = 51 (mod 97), 


x = 10 (mod 95). 


It can be shown (see Exercise 53) that 537,140 is the unique nonnegative solution of this 
system less than 89,403,930. Consequently, 537,140 is the sum. Note that it is only when we 
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EXAMPLE 9 
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have to recover the integer represented by (65, 2, 51, 10) that we have to do arithmetic with 
integers larger than 100. < 


Particularly good choices for moduli for arithmetic with large integers are sets of integers of 
the form 2* — 1, where k is a positive integer, because it is easy to do binary arithmetic modulo 
such integers, and because it is easy to find sets of such integers that are pairwise relatively 
prime. [The second reason is a consequence of the fact that gcd(2¢ — 1, 2? — 1) = 2846 b) — 1, 
as Exercise 37 in Section 4.3 shows.] Suppose, for instance, that we can do arithmetic with 
integers less than 2*° easily on our computer, but that working with larger integers requires 
special procedures. We can use pairwise relatively prime moduli less than 2% to perform 
arithmetic with integers as large as their product. For example, as Exercise 38 in Section 
4.3 shows, the integers 2% — 1, 2% — 1, 23 — 1,23! — 1,2? — 1, and 2? — 1 are pairwise 
relatively prime. Because the product of these six moduli exceeds 2!*4, we can perform 
arithmetic with integers as large as 2'** (as long as the results do not exceed this number) by 
doing arithmetic modulo each of these six moduli, none of which exceeds 2°». 


4.4.5 Fermat’s Little Theorem 


The French mathematician Pierre de Fermat, one of the leading mathematicians of the first half 
of the 17th century, made many important discoveries in number theory. One of the most useful 
of these states that p divides a?-' — 1 whenever p is prime and a is an integer not divisible by 
p. Fermat announced this result in a letter to one of his correspondents. However, he did not 
include a proof in the letter, stating that he feared the proof would be too long. Although Fermat 
never published a proof of this fact, there is little doubt that he knew how to prove it, unlike the 
result known as Fermat’s last theorem. The first published proof is credited to Leonhard Euler. 
We now State this theorem in terms of congruences. 


FERMAT?’S LITTLE THEOREM If p is prime and a is an integer not divisible by p, 
then 


a?-! = 1 (mod p). 
Furthermore, for every integer a we have 


a = a (mod p). 


Remark: Fermat’s little theorem tells us that if a € Z 


P-l~]j 
p then aP™ = 1 in Z, 


The proof of Theorem 3 is outlined in Exercise 19. 
Fermať’s little theorem is extremely useful in computing the remainders modulo p of large 
powers of integers, as Example 9 illustrates. 


Find 7222 mod 11. 


Solution: We can use Fermat’s little theorem to evaluate 77? mod 11 rather than using the fast 
modular exponentiation algorithm. By Fermat’s little theorem we know that 7!° = 1 (mod 11), 
so (7!°)* = 1 (mod 11) for every positive integer k. To take advantage of this last congruence, 
we divide the exponent 222 by 10, finding that 222 = 22 - 10 + 2. We now see that 


PE = 7710+ =P = (1)? +49 = 5 (mod 11). 


It follows that 7222 mod 11 = 5. <4 
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Example 9 illustrated how we can use Fermat's little theorem to compute a” mod p, where 
p is prime and p J a. First, we use the division algorithm to find the quotient q and remainder r 
when n is divided by p — 1, so that n = g(p — 1) + r, where 0 < r < p — 1. It follows that a” = 
a1P-D+r — (gP-!)Iq" = 14a" = a" (mod p). Hence, to find a” mod p, we only need to compute 
a’ mod p. We will take advantage of this simplification many times in our study of number 
theory. 


4.4.6 Pseudoprimes 


In Section 4.2 we showed that an integer n is prime when it is not divisible by any prime p with 
p< y/n. Unfortunately, using this criterion to show that a given integer is prime is inefficient. 


It requires that we find all primes not exceeding yn and that we carry out trial division by each 
such prime to see whether it divides n. 

Are there more efficient ways to determine whether an integer is prime? According to some 
sources, ancient Chinese mathematicians believed that n was an odd prime if and only if 


2"-! = ] (mod n). 


If this were true, it would provide an efficient primality test. Why did they believe this 
congruence could be used to determine whether an integer n > 2 is prime? First, they observed 
that the congruence holds whenever n is an odd prime. For example, 5 is prime and 


257! = 24 = 16 = 1 (mod 5). 


By Fermat’s little theorem, we know that this observation was correct, that is, ie 
1 (mod n) whenever n is an odd prime. Second, they never found a composite integer n for 
which the congruence holds. However, the ancient Chinese were only partially correct. They 
were correct in thinking that the congruence holds whenever n is prime, but they were incorrect 
in concluding that n is necessarily prime if the congruence holds. 

Unfortunately, there are composite integers n such that 2”~! = 1 (mod n). Such integers are 
called pseudoprimes to the base 2. 


The integer 341 is a pseudoprime to the base 2 because it is composite (341 = 11 - 31) and as 
Exercise 37 shows 


23% = 1 (mod 341). 4 


We can use an integer other than 2 as the base when we study pseudoprimes. 


PIERRE DE FERMAT (1601-1665) Pierre de Fermat, one of the most important mathematicians of the sev- 
enteenth century, was a lawyer by profession. He is the most famous amateur mathematician in history. Fermat 
published little of his mathematical discoveries. It is through his correspondence with other mathematicians 
that we know of his work. Fermat was one of the inventors of analytic geometry and developed some of the 
fundamental ideas of calculus. Fermat, along with Pascal, gave probability theory a mathematical basis. Fer- 
mat formulated what was the most famous unsolved problem in mathematics. He asserted that the equation 
x” + y” = z” has no nontrivial positive integer solutions when n is an integer greater than 2. For more than 300 
years, no proof (or counterexample) was found. In his copy of the works of the ancient Greek mathematician 
Diophantus, Fermat wrote that he had a proof but that it would not fit in the margin. Because the first proof, 
found by Andrew Wiles in 1994, relies on sophisticated, modern mathematics, most people think that Fermat 
thought he had a proof, but that the proof was incorrect. However, he may have been tempting others to look 
for a proof, not being able to find one himself. 
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Let b be a positive integer. If n is a composite positive integer, and b"! = 1 (mod n), then n 
is called a pseudoprime to the base b. 


Given a positive integer n, determining whether 2”7! = 1 (mod n) is a useful test that pro- 
vides some evidence concerning whether n is prime. In particular, if n satisfies this congruence, 
then it is either prime or a pseudoprime to the base 2; if n does not satisfy this congruence, it is 
composite. We can perform similar tests using bases b other than 2 and obtain more evidence 
as to whether n is prime. If n passes all such tests, it is either prime or a pseudoprime to all 
the bases b we have chosen. Furthermore, among the positive integers not exceeding x, where x 
is a positive real number, compared to primes there are relatively few pseudoprimes to the 
base b, where b is a positive integer. For example, among the positive integers less than 10!° 
there are 455,052,512 primes, but only 14,884 pseudoprimes to the base 2. Unfortunately, we 
cannot distinguish between primes and pseudoprimes just by choosing sufficiently many bases, 
because there are composite integers n that pass all tests with bases b such that gcd(b, n) = 1. 
This leads to Definition 2. 


A composite integer n that satisfies the congruence b"! = 1 (mod n) for all positive integers b 
with gcd(b, n) = 1 is called a Carmichael number. (These numbers are named after Robert 
Carmichael, who studied them in the early twentieth century.) 


The integer 561 is a Carmichael number. To see this, first note that 561 is composite 
because 561 = 3-11-17. Next, note that if gcd(b, 561) = 1, then gcd(b, 3) = gcd(b, 11) = 
gcd(b, 17) = 1. 

Using Fermat’s little theorem we find that 


b? = 1 (mod 3), b!? = 1 (mod 11), and b!® = 1 (mod 17). 
It follows that 


p560 = (b*)789 = 1 (mod 3), 
b560 = (b!°)°6 = 1 (mod 11), 
5° = (b'°)* = 1 (mod 17). 


By Exercise 29, it follows that b56? =1 (mod 561) for all positive integers b with 
gcd(b, 561) = 1. Hence, 561 is a Carmichael number. < 


Although there are infinitely many Carmichael numbers, more delicate tests, described in 
the exercise set, can be devised that can be used as the basis for efficient probabilistic primality 
tests. Such tests can be used to quickly show that it is almost certainly the case that a given 


ROBERT DANIEL CARMICHAEL (1879-1967) Robert Daniel Carmichael was born in Alabama. He re- 
ceived his undergraduate degree from Lineville College in 1898 and his Ph.D. in 1911 from Princeton. 
Carmichael held positions at Indiana University from 1911 until 1915 and at the University of Illinois from 
1915 until 1947. Carmichael was an active researcher in a wide variety of areas, including number theory, real 
analysis, differential equations, mathematical physics, and group theory. His Ph.D. thesis, written under the di- 
rection of G. D. Birkhoff, is considered the first significant American contribution to the subject of differential 
equations. 
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Definition 3 


EXAMPLE 12 


Definition 4 


EXAMPLE 13 


a 


The discrete logarithm 
problem is hard! 


integer is prime. More precisely, if an integer is not prime, then the probability that it passes a 
series of tests is close to 0. We will describe such a test in Chapter 7 and discuss the notions 
from probability theory that this test relies on. These probabilistic primality tests can be used, 
and are used, to find large primes extremely rapidly on computers. 


4.4.7 Primitive Roots and Discrete Logarithms 


In the set of positive real numbers, if b > 1, and x = b’, we say that y is the logarithm of x to 
the base b. Here, we will show that we can also define the concept of logarithms modulo p of 
positive integers, where p is a prime. Before we do so, we need a definition. 


A primitive root modulo a prime p is an integer r in Z, such that every nonzero element of 
Z, is a power of r. 


Determine whether 2 and 3 are primitive roots modulo 11. 


Solution: When we compute the powers of 2 in Z,,, we obtain 2! = 2, 2? = 4, 2? = 8, 24 = 5, 
25 = 10, 2° = 9, 27 =7, 28 = 3, 2? = 6, 2! = 1. Because every nonzero element of Z,, is a 
power of 2, 2 is a primitive root of 11. 

When we compute the powers of 3 modulo 11, we obtain 3! = 3, 3? = 9, 33 = 5, 34 = 4, 
35 = 1. This pattern repeats when we compute higher powers of 3. Because not all nonzero 
elements of Z,, are powers of 3, we conclude that 3 is not a primitive root of 11. < 


An important fact in number theory is that there is a primitive root modulo p for every 
prime p. We refer the reader to [Ro10] for a proof of this fact. Suppose that p is prime and r is 
a primitive root modulo p. If a is an integer between 1 and p — 1, that is, a nonzero element of 
Z, we know that there is an unique exponent e such that r° = a in Z, that is, r° mod p = a. 


Suppose that p is a prime, r is a primitive root modulo p, and a is an integer between 1 and 
p — 1 inclusive. If r° mod p = a and 0 < e < p — 1, we say that e is the discrete logarithm of 
a modulo p to the base r and we write log, a = e (where the prime p is understood). 


Find the discrete logarithms of 3 and 5 modulo 11 to the base 2. 


Solution: When we computed the powers of 2 modulo 11 in Example 12, we found that 28 = 3 
and 24 = 5 in Z,,. Hence, the discrete logarithms of 3 and 5 modulo 11 to the base 2 are 8 
and 4, respectively. (These are the powers of 2 that equal 3 and 5, respectively, in Z,,.) We 
write log, 3 = 8 and log, 5 = 4 (where the modulus 11 is understood and not explicitly noted 
in the notation). 


The discrete logarithm problem takes as input a prime p, a primitive root r modulo p, 
and a positive integer a E Z,; its output is the discrete logarithm of a modulo p to the base 
r. Although this problem might seem not to be that difficult, it turns out that no polynomial 
time algorithm is known for solving it. The difficulty of this problem plays an important role in 
cryptography, as we will see in Section 4.6. 


Exercises 


+T: 


10. 


11. 


12. 


13. 


14. 


. Show that 15 is an inverse of 7 modulo 26. 
. Show that 937 is an inverse of 13 modulo 2436. 
. By inspection (as discussed prior to Example 1), find an 


inverse of 4 modulo 9. 


. By inspection (as discussed prior to Example 1), find an 


inverse of 2 modulo 17. 


. Find an inverse of a modulo m for each of these pairs 


of relatively prime integers using the method followed in 
Example 2. 

a) a=4,m=9 

b) a=19,m= 141 

c) a=55,m= 89 

d) a= 89, m = 232 


. Find an inverse of a modulo m for each of these pairs 


of relatively prime integers using the method followed in 
Example 2. 

a) a=2,m=17 

b) a = 34, m = 89 

c) a = 144, m = 233 

d) a = 200, m = 1001 

Show that if a and m are relatively prime positive inte- 
gers, then the inverse of a modulo m is unique modulo 
m. [Hint: Assume that there are two solutions b and c of 
the congruence ax = 1 (mod m). Use Theorem 7 of Sec- 
tion 4.3 to show that b = c (mod m).] 


. Show that an inverse of a modulo m, where a is an in- 


teger and m > 2 is a positive integer, does not exist if 
gcd(a, m) > 1. 


. Solve the congruence 4x = 5 (mod 9) using the inverse 


of 4 modulo 9 found in part (a) of Exercise 5. 

Solve the congruence 2x = 7 (mod 17) using the inverse 
of 2 modulo 17 found in part (a) of Exercise 6. 

Solve each of these congruences using the modular in- 
verses found in parts (b), (c), and (d) of Exercise 5. 

a) 19x = 4 (mod 141) 

b) 55x = 34 (mod 89) 

c) 89x = 2 (mod 232) 

Solve each of these congruences using the modular in- 
verses found in parts (b), (c), and (d) of Exercise 6. 

a) 34x = 77 (mod 89) 

b) 144x = 4 (mod 233) 

c) 200x = 13 (mod 1001) 

Find the solutions of the congruence 15x? + 19x = 5 
(mod 11). [Hint: Show the congruence is equivalent to 
the congruence 15x? + 19x + 6 = 0 (mod 11). Factor the 
left-hand side of the congruence; show that a solution of 
the quadratic congruence is a solution of one of the two 
different linear congruences. ] 

Find the solutions of the congruence 12x* + 25x = 
10 (mod 11). [Hint: Show the congruence is equivalence 
to the congruence 12x” + 25x + 12 = 0 (mod 11). Factor 
the left-hand side of the congruence; show that a solu- 
tion of the quadratic congruence is a solution of one of 
two different linear congruences. ] 


15. 


16. 


17. 


* 18. 


*19, 


20. 


21. 


22. 


23. 


24. 
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Show that if m is an integer greater than 1 and ac= 
bc (mod m), then a = b (mod m/gcd(c, m)). 


a) Show that the positive integers less than 11, except 
1 and 10, can be split into pairs of integers such that 
each pair consists of integers that are inverses of each 
other modulo 11. 


b) Use part (a) to show that 10! = —1 (mod 11). 


Show that if p is prime, the only solutions of x? = 
1 (mod p) are integers x such that x = 1 (mod p) orx = —1 
(mod p). 


a) Generalize the result in part (a) of Exercise 16; that 
is, show that if p is a prime, the positive integers less 
than p, except 1 and p — 1, can be split into (p — 3)/2 
pairs of integers such that each pair consists of inte- 
gers that are inverses of each other. [Hint: Use the 
result of Exercise 17.] 

b) From part (a) conclude that (p — 1)! = —1 (mod p) 
whenever p is prime. This result is known as Wilson’s 
theorem. 

c) What can we conclude if n is a positive integer such 
that (n — 1)! # —1 (mod n)? 

This exercise outlines a proof of Fermat’s little theorem. 

a) Suppose that a is not divisible by the prime p. Show 
that no two of the integers 1 - a, 2-a,...,(p — 1)a are 
congruent modulo p. 

b) Conclude from part (a) that the product of 
1,2, ...,p — 1 is congruent modulo p to the product 
of a, 2a, ..., (p — 1)a. Use this to show that 


(p — 1)! = a”™! (p — 1)! (mod p). 


c) Use Theorem 7 of Section 4.3 to show from part (b) 
that a?-! = 1 (mod p) if p J a. [Hint: Use Lemma 3 
of Section 4.3 to show that p does not divide (p — 1)! 
and then use Theorem 7 of Section 4.3. Alternatively, 
use Wilson’s theorem from Exercise 18(b).] 

d) Use part (c) to show that a? = a (mod p) for all inte- 
gers a. 

Use the construction in the proof of the Chinese remain- 

der theorem to find all solutions to the system of congru- 

ences x = 2 (mod 3), x = 1 (mod 4), and x = 3 (mod 5). 


Use the construction in the proof of the Chinese remain- 
der theorem to find all solutions to the system of congru- 
ences x = 1 (mod 2), x = 2 (mod 3), x = 3 (mod 5), and 
x = 4 (mod 11). 


Solve the system of congruence x =3 (mod 6) and 
x = 4 (mod 7) using the method of back substitution. 


Solve the system of congruences in Exercise 20 using the 
method of back substitution. 


Solve the system of congruences in Exercise 21 using the 
method of back substitution. 


302 


25. 


* 26. 


* 27: 


28. 


*29. 


*30. 


31. 
32. 
33. 
34. 
35. 


36. 
37. 


38. 


39. 


40. 


41. 
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Write out in pseudocode an algorithm for solving a 
simultaneous system of linear congruences based on 
the construction in the proof of the Chinese remainder 
theorem. 

Find all solutions, if any, to the system of congruences 

x = 5 (mod 6), x = 3 (mod 10), and x = 8 (mod 15). 

Find all solutions, if any, to the system of congruences 

x = 7 (mod 9), x = 4 (mod 12), and x = 16 (mod 21). 

Use the Chinese remainder theorem to show that an in- 

teger a, with 0 < a < m = m,m, ---m,, where the posi- 

tive integers m,, m, ...,m, are pairwise relatively prime, 

can be represented uniquely by the n-tuple (a mod m, 

a mod m), ...,a mod m,). 

Let m, m3, ..., m, be pairwise relatively prime inte- 

gers greater than or equal to 2. Show that if a= 

b (mod m,) for i = 1, 2, ... , n, then a = b (mod m), where 

m =m My + m,,. (This result will be used in Exercise 30 

to prove the Chinese remainder theorem. Consequently, 

do not use the Chinese remainder theorem to prove it.) 

Complete the proof of the Chinese remainder theorem 

by showing that the simultaneous solution of a system 

of linear congruences modulo pairwise relatively prime 
moduli is unique modulo the product of these moduli. 

[Hint: Assume that x and y are two simultaneous solu- 

tions. Show that m; | x — y for all i. Using Exercise 29, 

conclude that m = m;m, = m, | x — y.] 

Which integers leave a remainder of 1 when divided by 2 

and also leave a remainder of 1 when divided by 3? 

Which integers are divisible by 5 but leave a remainder 

of 1 when divided by 3? 

Use Fermat’s little theorem to find 7!?! mod 13. 

Use Fermat’s little theorem to find 23!°°? mod 41. 

Use Fermat’s little theorem to show that if p is prime and 

p | a, then a?~? is an inverse of a modulo p. 

Use Exercise 35 to find an inverse of 5 modulo 41. 

a) Show that 234° = 1 (mod 11) by Fermat’s little theo- 
rem and noting that 2349 = (2!0)34, 

b) Show that 2*4° = 1 (mod 31) using the fact that 2*4° = 
(25) 68 = 3268, 

c) Conclude from parts (a) and (b) that 2% = 
1 (mod 341). 

a) Use Fermat’s little theorem to compute 3%? mod 5, 

33 mod 7, and 3° mod 11. 

b) Use your results from part (a) and the Chinese re- 

mainder theorem to find 3°°* mod 385. (Note that 

385 =5-7-11.) 

a) Use Fermat’s little theorem to compute 52003 mod 7, 

57003 mod 11, and 57° mod 13. 

b) Use your results from part (a) and the Chinese re- 
mainder theorem to find 57°? mod 1001. (Note that 
1001 =7-11- 13.) 

Show with the help of Fermat’s little theorem that if n is 

a positive integer, then 42 divides n’ — n. 

Show that if p is an odd prime, then every divisor of the 

Mersenne number 2? — 1 is of the form 2kp + 1, where k 

is a nonnegative integer. [Hint: Use Fermat’s little theo- 

rem and Exercise 37 of Section 4.3.] 


42. Use Exercise 41 to determine whether M,, = 2°? — 1 = 
8191 and M; = 2” — 1 = 8,388,607 are prime. 

43. Use Exercise 41 to determine whether M}; = 2!! — 1 = 
2047 and M,, = 2" — 1 = 131,071 are prime. 


E Let n be a positive integer and let n — 1 = 2°t, where s is a 


nonnegative integer and ¢ is an odd positive integer. We say 

that n passes Miller’s test for the base b if either b‘ = 1 (mod 

n) or b” = —1 (mod n) for some j with 0 < j < s — 1.Itcanbe 

shown (see [Ro10]) that a composite integer n passes Miller’s 

test for fewer than n/4 bases b with 1 < b <n. A compos- 
ite positive integer n that passes Miller’s test to the base b is 

called a strong pseudoprime to the base b. 

* 44, Show that if n is prime and b is a positive integer with 
n | b, then n passes Miller’s test to the base b. 

45. Show that 2047 is a strong pseudoprime to the base 2 by 
showing that it passes Miller’s test to the base 2, but is 
composite. 

46. Show that 1729 is a Carmichael number. 

47. Show that 2821 is a Carmichael number. 

*48, Show that if n = pipz + Py, where pj, Po, ..., Py are dis- 
tinct primes that satisfy Dj - 1|n-1 for j =1,2,...,k, 
then n is a Carmichael number. 

49. a) Use Exercise 48 to show that every integer of the 
form (6m + 1)(12m + 1)(18m + 1), where m is a pos- 
itive integer and 6m +1, 12m + 1, and 18m + 1 are 
all primes, is a Carmichael number. 

b) Use part (a) to show that 172,947,529 is a Car- 
michael number. 

50. Find the nonnegative integer a less than 28 represented by 
each of these pairs, where each pair represents (a mod 4, 


amod 7). 

a) (0, 0) b) (1, 0) c) (1,1) 
d) (2, 1) e) (2,2) f) (0,3) 
g) (2,0) h) (3,5) i) (3,6) 


51. Express each nonnegative integer a less than 15 as a pair 
(a mod 3, a mod 5). 

52. Explain how to use the pairs found in Exercise 51 to 
add 4 and 7. 

53. Solve the system of congruences that arises in Example 8. 

54. Show that 2 is a primitive root of 19. 

55. Find the discrete logarithms of 5 and 6 to the base 2 mod- 
ulo 19. 

56. Let p be an odd prime and r a primitive root of p. Show 
that if a and b are positive integers in Z,, then log,(ab) = 
log„a + log, b (mod p — 1). 

57. Write out a table of discrete logarithms modulo 17 with 
respect to the primitive root 3. 

If m is a positive integer, the integer a is a quadratic residue 

of m if gcd(a, m) = 1 and the congruence x2 = a (mod m) has 

a solution. In other words, a quadratic residue of m is an 

integer relatively prime to m that is a perfect square mod- 

ulo m. If a is not a quadratic residue of m and gcd(a, m) = 1, 

we say that it is a quadratic nonresidue of m. For exam- 

ple, 2 is a quadratic residue of 7 because gcd(2, 7) = 1 and 

3? = 2(mod 7) and 3 is a quadratic nonresidue of 7 because 

gcd(3, 7) = 1 and x? = 3 (mod 7) has no solution. 
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58. Which integers are quadratic residues of 11? [Hint: If a is a quadratic residue modulo p, apply Fer- 
59. Show that if p is an odd prime and a is an integer not mat’s little theorem; otherwise, apply Wilson’s theorem, 
divisible by p, then the congruence x? = a (mod p) has given in Exercise 18(b).] 
either no solutions or exactly two incongruent solutions 63. Use Exercise 62 to show that if p is an odd prime and a 
modulo p. and b are integers not divisible by p, then 
60. Show that if p is an odd prime, then there are exactly 
(p — 1)/2 quadratic residues of p among the integers (2) _ (2) (2) 
1,2,...,p—1. p p p i 


If p is an odd prime and a is an integer not divisible by p, the 
64. Show that if p is an odd prime, then —1 is a quadratic 


a\. ae : 
Legendre symbol (2) is defined to be 1 if a is a quadratic residue of p if p = 1 (mod4), and —1 is not a quadratic 


residue of p and —1 ET residue of p if p = 3 (mod 4). [Hint: Use Exercise 62.] 
61. Show that if p is an odd prime and a and b are integers 65. Find all solutions of the congruence x? = 29 (mod 35). 
with a = b (mod p), then [Hint: Find the solutions of this congruence modulo 
5 and modulo 7, and then use the Chinese remainder 
a b theorem.] 
(2) ~ (2) i 66. Find all solutions of the congruence x? = 16 (mod 105). 
[Hint: Find the solutions of this congruence modulo 3, 
62. Prove Euler’s criterion, which states that if p is an odd modulo 5, and modulo 7, and then use the Chinese re- 
prime and a is a positive integer not divisible by p, then mainder theorem. ] 
67. Describe a brute-force algorithm for solving the discrete 
(£: ) = g-)? (mo dp). logarithm problem and find the worst-case and average- 
case time complexity of this algorithm. 


Applications of Congruences 


Congruences have many applications to discrete mathematics, computer science, and many 
other disciplines. We will introduce three applications in this section: the use of congruences 
to assign memory locations to computer files, the generation of pseudorandom numbers, and 
check digits. 

Suppose that a customer identification number is ten digits long. To retrieve customer files 
quickly, we do not want to assign a memory location to a customer record using the ten-digit 
identification number. Instead, we want to use a smaller integer associated to the identification 
number. This can be done using what is known as a hashing function. In this section we will 
show how we can use modular arithmetic to do hashing. 

Constructing sequences of random numbers is important for randomized algorithms, for 
simulations, and for many other purposes. Constructing a sequence of truly random numbers 
is extremely difficult, or perhaps impossible, because any method for generating what are sup- 
posed to be random numbers may generate numbers with hidden patterns. As a consequence, 
methods have been developed for finding sequences of numbers that have many desirable prop- 
erties of random numbers, and which can be used for various purposes in place of random 
numbers. In this section we will show how to use congruences to generate sequences of pseu- 
dorandom numbers. The advantage is that the pseudorandom numbers so generated are con- 
structed quickly; the disadvantage is that they have too much predictability to be used for many 
tasks. 

Congruences also can be used to produce check digits for identification numbers of various 
kinds, such as code numbers used to identify retail products, numbers used to identify books, 
airline ticket numbers, and so on. We will explain how to construct check digits using con- 
gruences for a variety of types of identification numbers. We will show that these check digits 
can be used to detect certain kinds of common errors made when identification numbers are 
printed. 
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EXAMPLE 1 


EXAMPLE 2 


4.5.1 Hashing Functions 


The central computer at an insurance company maintains records for each of its customers. 
How can memory locations be assigned so that customer records can be retrieved quickly? The 
solution to this problem is to use a suitably chosen hashing function. Records are identified 
using a key, which uniquely identifies each customer’s records. For instance, customer records 
are often identified using the Social Security number of the customer as the key. A hashing 
function 4 assigns memory location h(x) to the record that has k as its key. 

In practice, many different hashing functions are used. One of the most common is the 
function 


h(k) = k mod m 


where m is the number of available memory locations. 

Hashing functions should be easily evaluated so that files can be quickly located. The hash- 
ing function h(k) = k mod m meets this requirement; to find h(k), we need only compute the 
remainder when k is divided by m. Furthermore, the hashing function should be onto, so that 
all memory locations are possible. The function h(k) = k mod m also satisfies this property. 


Find the memory locations assigned by the hashing function h(k) = k mod 111 to the records 
of customers with Social Security numbers 064212848 and 037149212. 


Solution: The record of the customer with Social Security number 064212848 is assigned to 
memory location 14, because 


h(064212848) = 064212848 mod 111 = 14. 
Similarly, because 
h(037149212) = 037149212 mod 111 = 65, 


the record of the customer with Social Security number 037149212 is assigned to memory 
location 65. < 


Because a hashing function is not one-to-one (because there are more possible keys than 
memory locations), more than one file may be assigned to a memory location. When this hap- 
pens, we say that a collision occurs. One way to resolve a collision is to assign the first free 
location following the occupied memory location assigned by the hashing function. 


After making the assignments of records to memory locations in Example 1, assign a memory 
location to the record of the customer with Social Security number 107405723. 


Solution: First note that the hashing function h(k) = kmod 111 maps the Social Security number 
107405723 to location 14, because 


h(107405723) = 107405723 mod 111 = 14. 


However, this location is already occupied (by the file of the customer with Social Security 
number 064212848). But, because memory location 15, the first location following memory lo- 
cation 14, is free, we assign the record of the customer with Social Security number 107405723 
to this location. < 


Links » 
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In Example 1 we used a linear probing function, namely, h(k, i) = h(k) +i mod m, to look 
for the first free memory location, where i runs from 0 to m — 1. There are many other ways to 
resolve collisions that are discussed in the references on hashing functions given at the end of 
the book. 


4.5.2 Pseudorandom Numbers 


Randomly chosen numbers are often needed for computer simulations. Different methods have 
been devised for generating numbers that have properties of randomly chosen numbers. Because 
numbers generated by systematic methods are not truly random, they are called pseudorandom 
numbers. 

The most commonly used procedure for generating pseudorandom numbers is the 
linear congruential method. We choose four integers: the modulus m, multiplier a, 
increment c, and seed x, with 2 < a < m,0 < c < m, and 0 < xy < m. We generate a sequence 
of pseudorandom numbers {x,}, with O < x, < m for all n, by successively using the recursively 
defined function 


X +1 = (ax, +c) mod m. 


(This is an example of a recursive definition, discussed in Section 5.3. In that section we will 
show that such sequences are well defined.) 

Many computer experiments require the generation of pseudorandom numbers between 0 
and 1. To generate such numbers, we divide numbers generated with a linear congruential gen- 
erator by the modulus: that is, we use the numbers x,, /m. 


Find the sequence of pseudorandom numbers generated by the linear congruential method with 
modulus m = 9, multiplier a = 7, increment c = 4, and seed x, = 3. 


Solution: We compute the terms of this sequence by successively using the recursively defined 
function x,,,; = (7x, +4) mod 9, beginning by inserting the seed xy = 3 to find x,. We find that 


xı = 7x) +4 mod 9 = 7-3+4 mod 9 = 25 mod 9 = 7, 
X, = 7x, +4mod 9 =7-7+4 mod 9 = 53 mod 9 = 8, 
x, = 7x, + 4 mod 9 =7-8+4 mod 9 = 60 mod 9 = 6, 
x4 = 7x, + 4 mod 9 =7-6+4 mod 9 = 46 mod 9 = 1, 
x; = 7x, +4 mod 9 =7-1+4 mod 9 = 11 mod 9 = 2, 
x6 = 7x; + 4 mod 9 =7-2+4 mod 9 = 18 mod 9 = 0, 
xX =7x%,+4mod9=7-0+4mo0d9= 4mod9=4, 
Xg = 7x, + 4 mod 9 =7-4+4 mod 9 = 32 mod 9 = 5, 
Xo = 7xg +4 mod 9 = 7-5 +4 mod 9 = 39 mod 9 = 3. 


Because xg = X) and because each term depends only on the previous term, we see that the 
sequence 


3, 7, 8, 6, 1, 2, 0, 4, 5, 3, 7, 8, 6, 1, 2, 0, 4, 5, 3, ... 


is generated. This sequence contains nine different numbers before repeating. < 


Most computers do use linear congruential generators to generate pseudorandom numbers. 
Often, a linear congruential generator with increment c = 0 is used. Such a generator is called 
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EXAMPLE 4 


EXAMPLE 5 


a pure multiplicative generator. For example, the pure multiplicative generator with modulus 
23! — 1 and multiplier P = 16,807 is widely used. With these values, it can be shown that 231 
2 numbers are generated before repetition begins. 

Pseudorandom numbers generated by linear congruential generators have long been used 
for many tasks. Unfortunately, it has been shown that sequences of pseudorandom numbers gen- 
erated in this way do not share some important statistical properties that true random numbers 
have. Because of this, it is not advisable to use them for some tasks, such as large simulations. 
For such sensitive tasks, other methods are used to produce sequences of pseudorandom num- 
bers, either using some sort of algorithm or sampling numbers arising from a random physical 
phenomenon. For more details on pseudorandom number, see [Kn97] and [Re10]. 


4.5.3 Check Digits 


Congruences are used to check for errors in digit strings. A common technique for detecting 
errors in such strings is to add an extra digit at the end of the string. This final digit, or check digit, 
is calculated using a particular function. Then, to determine whether a digit string is correct, a 
check is made to see whether this final digit has the correct value. We begin with an application 
of this idea for checking the correctness of bit strings. 


Parity Check Bits Digital information is represented by bit string, split into blocks of a spec- 
ified size. Before each block is stored or transmitted, an extra bit, called a parity check bit, can 
be appended to each block. The parity check bit x,,,, for the bit string x,x, ...x,, is defined by 


Xn =X, +X + +++ +x, mod 2. 


It follows that x,,,, is O if there are an even number of 1 bits in the block of n bits and it is 1 if 
there are an odd number of 1 bits in the block of n bits. When we examine a string that includes 
a parity check bit, we know that there is an error in it if the parity check bit is wrong. However, 
when the parity check bit is correct, there still may be an error. A parity check can detect an odd 
number of errors in the previous bits, but not an even number of errors. (See Exercise 14.) 

Suppose we receive in a transmission the bit strings 01100101 and 11010110, each ending 
with a parity check bit. Should we accept these bit strings as correct? 


Solution: Before accepting these strings as correct, we examine their parity check bits. The 
parity check bit of the first string is 1. Because 0 + 1 +1+0+0+1+0=1 (mod 2), the parity 
check bit is correct. The parity check bit of the second string is 0. We find that 1 +1 +0+ 1+ 
0+1+1+=1 (mod 2), so the parity check is incorrect. We conclude that the first string may 
have been transmitted correctly and we know for certain that the second string was transmitted 
incorrectly. We accept the first string as correct (even though it still may contain an even number 
of errors), but we reject the second string. < 


Check bits computed using congruences are used extensively to verify the correctness of 
various kinds of identification numbers. Examples 5 and 6 show how check bits are computed 
for codes that identify products (Universal Product Codes) and books (International Standard 
Book Numbers). The preambles to Exercises 18, 28, and 32 introduce the use of congruences 
to find and use check digits in money order numbers, airline ticket numbers, and identification 
numbers for periodicals, respectively. Note that congruences are also used to compute check 
digits for bank account numbers, drivers license numbers, credit card numbers, and many other 
types of identification numbers. 


UPCs Retail products are identified by their Universal Product Codes (UPCs). The most 
common form of a UPC has 12 decimal digits: the first digit identifies the product category, the 


EXAMPLE 6 


Remember that the 
check digit of an 
ISBN-10 can be an X! 
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next five digits identify the manufacturer, the following five identify the particular product, and 
the last digit is a check digit. The check digit is determined by the congruence 


3X, +x + 3x3 + x4 + 3x5 + x6 + 3x7 + Xg + 3X + x10 + 3X1, +X. = 0 (mod 10). 


Answer these questions: 


(a) Suppose that the first 11 digits of a UPC are 79357343104. What is the check digit? 
(b) Is 041331021641 a valid UPC? 


Solution: (a) We insert the digits of 79357343104 into the congruence for UPC check digits. This 
gives3-7+9+3-34+54+3-743+4+3-44+3+4+3-14+0+3-4+4x,) = 0(mod 10). Simpli- 
fying, we have 21 +9+9+5+21+3+12+3+3+0+ 12+ x, =0(mod 10). Hence, 
98 +x), = 0 (mod 10). It follows that x,, = 2 (mod 10), so the check digit is 2. 


(b) To check whether 041331021641 is valid, we insert the digits into the congruence these 
digits must satisfy. This gives3-0+44+3-14+343-34+14+3-04+24+3-14+6+3-44+1 
=0+4+3+3+9+1+0+2+3+6+12+ 1 = 4 £0(mod 10). Hence, 041331021641 is 
not a valid UPC. < 


ISBNs All books are identified by an International Standard Book Number (ISBN-10), a 
10-digit code x,x, ... X1ọ, assigned by the publisher. (Recently, a 13-digit code known as ISBN- 
13 was introduced to identify a larger number of published works; see the preamble to Exercise 
42 in the Supplementary Exercises.) An ISBN-10 consists of blocks identifying the language, 
the publisher, the number assigned to the book by its publishing company, and finally, a check 
digit that is either a digit or the letter X (used to represent 10). This check digit is selected so 
that 


9 


Xo = >) ix; (mod 11), 
i=! 


or equivalently, so that 


10 
>, ix, = 0 (mod 11). 


i=1 
Answer these questions about ISBN-10s: 


(a) The first nine digits of the ISBN-10 of the sixth edition of this book are 007288008. What 
is the check digit? 


(b) Is 084930149X a valid ISBN-10? 


Solution: (a) The check digit is determined by the congruence a ix; = 0 (mod 11). Inserting 
the digits 007288008 gives xo =1-0+2-04+3-74+4-24+5-8+6-8+7-0+8-0+9:- 
8 (mod 11). This means that xo =0+0+21+8+40+48+0+0+472 (mod 11), so xio = 
189 = 2 (mod 11). Hence, x, = 2. 


(b) To see whether 084930149X is a valid ISBN-10, we see if 5e ix; = 0 (mod 11). We 
see that 1-0+2-8+3-4+4-9+5-3+6-0+7-1+8-4+9-9+10-10 = 0+16+ 
12+ 364+ 15+04+7+32 +81 + 100 = 299 = 2 ~ 0 (mod 11). Hence, 084930149X is not a 
valid ISBN-10. < 
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Publishers sometimes 
do not calculate ISBNs 
correctly for their 
books, as was done for 
an earlier edition of this 
text. 


Several kinds of errors often arise in identification numbers. A single error, an error in one 
digit of an identification number, is perhaps the most common type of error. Another common 
kind of error is a transposition error, which occurs when two digits are accidentally inter- 
changed. For each type of identification number, including a check digit, we would like to be 
able to detect these common types of errors, as well as other types of errors. We will investigate 
whether the check digit for ISBNs can detect single errors and transposition errors. Whether 
check digits for UPCs can detect these kinds of errors is left as Exercises 26 and 27. 

Suppose that x; x, ...xX;9 is a valid ISBN (so that pe x; = 0 (mod 10)). We will show that 
we can detect a single error and a transposition of two digits (where we include the possibility 
that one of the two digits is the check digit X, representing 10). Suppose that this ISBN has been 
printed with a single error as y, y, ... yj. If there is a single error, then, for some integer j, y; = x; 
fori #j and y, = x; + a, where —10 <a < 10 anda # 0. Note that a = y; — x; is the error in the 
jth place. It then follows that 


10 10 


> iy; = ( È ix;) + ja = ja # 0 (mod 11). 


i=1 i=l 


These last two congruences hold because ye x; = 0(mod 10) and 11 fja, because 11 Yj and 
11 Ya. We conclude that y,y,...y,q is not a valid ISBN. So, we have detected the single 
error. 

Now suppose that two unequal digits have been transposed. It follows that there are distinct 
integers j and k such that y; = x, and y, = x;, and y; = x; for i # j and i # k. Hence, 


10 10 


> iy; = (YO ixi) + Gx — ix) + (oj — kx) = G- OO — xj) Æ 0 (mod 11), 


i=l i=l 


because 2 eer x; = 0 (mod 10) and 11 y (j — k) and 11 Y (xp — x;). We see that y1y2 ... Yio is not 
a valid ISBN. Thus, we can detect the interchange of two unequal digits. 


Exercises 
1. Which memory locations are assigned by the hashing Another way to resolve collisions in hashing is to use double 
function h(k) = k mod 97 to the records of insurance hashing. We use an initial hashing function h(k) = k mod p, 
company customers with these Social Security numbers? where p is prime. We also use a second hashing function 
a) 034567981 b) 183211232 g(k) = (k + 1)mod (p — 2). When a collision occurs, we use 
c) 220195744 d) 987255335 a probing sequence h(k, i) = (h(k) + i - g(k)) mod p. 
2. Which memory locations are assigned by the hashing 4. Use the double hashing procedure we have described 


function h(k) = k mod 101 to the records of insurance 

company customers with these Social Security numbers? 

a) 104578690 b) 432222187 

c) 372201919 d) 501338753 

. A parking lot has 31 visitor spaces, numbered from 0 to 

30. Visitors are assigned parking spaces using the hash- 

ing function h(k) = k mod 31, where k is the number 

formed from the first three digits on a visitor’s license 
plate. 

a) Which spaces are assigned by the hashing function to 
cars that have these first three digits on their license 
plates: 317, 918, 007, 100, 111, 310? 

b) Describe a procedure visitors should follow to find a 
free parking space, when the space they are assigned 
is occupied. 


with p = 4969 to assign memory locations to files for em- 
ployees with social security numbers kų = 132489971, 
k, = 509496993, k, = 546332190, ką = 034367980, 
k; = 047900151, kę = 329938157, k, = 212228844, 
kg = 325510778, ky = 353354519, ky) = 053708912. 


5. What sequence of pseudorandom numbers is gener- 
ated using the linear congruential generator x,,, = 
(3x, + 2) mod 13 with seed xọ = 1? 


6. What sequence of pseudorandom numbers is gener- 
ated using the linear congruential generator x,,, = 
(4x, + 1) mod 7 with seed x) = 3? 


7. What sequence of pseudorandom numbers is gener- 
ated using the pure multiplicative generator x,,, = 
3x,, mod 11 with seed xy = 2? 


8. Write an algorithm in pseudocode for generating a se- 
quence of pseudorandom numbers using a linear congru- 
ential generator. 

The middle-square method for generating pseudorandom 
numbers begins with an n-digit integer. This number is 
squared, initial zeros are appended to ensure that the result 
has 2n digits, and its middle n digits are used to form the next 
number in the sequence. This process is repeated to generate 
additional terms. 


9. Find the first eight terms of the sequence of four-digit 
pseudorandom numbers generated by the middle square 
method starting with 2357. 


10. Explain why both 3792 and 2916 would be bad choices 
for the initial term of a sequence of four-digit pseu- 
dorandom numbers generated by the middle square 
method. 

The power generator is a method for generating pseudo- 

random numbers. To use the power generator, parameters p 

and d are specified, where p is a prime, d is a positive integer 

such that p } d, and a seed x, is specified. The pseudorandom 
numbers x), X2, ... are generated using the recursive definition 

Xn = x4 mod p. 

11. Find the sequence of pseudorandom numbers generated 
by the power generator with p= 7, d =3, and seed 
Xp = 2. 


12. Find the sequence of pseudorandom numbers generated 
by the power generator with p = 11, d = 2, and seed 
Xp = 3. 


13. Suppose you received these bit strings over a communi- 
cations link, where the last bit is a parity check bit. In 
which string are you sure there is an error? 

a) 00000111111 b) 10101010101 
c) 11111100000 d) 10111101111 


14. Prove that a parity check bit can detect an error in a 
string if and only if the string contains an odd number of 
errors. 


15. The first nine digits of the ISBN-10 of the European ver- 
sion of the fifth edition of this book are 0-07-119881. 
What is the check digit for that book? 


16. The ISBN-10 of the sixth edition of Elementary Number 
Theory and Its Applications is 0-321-500Q1-8, where Q 
is a digit. Find the value of Q. 


17. Determine whether the check digit of the ISBN-10 for 
this textbook (the eighth edition of Discrete Mathemat- 
ics and Its Applications) was computed correctly by the 
publisher. 

The United States Postal Service (USPS) sells money orders 

identified by an 1 1-digit number x,x, ... xı. The first ten dig- 

its identify the money order; x,, is a check digit that satisfies 

X =X, +X +++ + Xio mod 9. 


4.5 Applications of Congruences 309 


18. Find the check digit for the USPS money orders that have 
identification number that start with these ten digits. 


a) 7555618873 b) 6966133421 
c) 8018927435 d) 3289744134 


19. Determine whether each of these numbers is a valid 
USPS money order identification number. 


a) 74051489623 b) 88382013445 
c) 56152240784 d) 66606631178 


20. One digit in each of these identification numbers of a 
postal money order is smudged. Can you recover the 
smudged digit, indicated by a Q, in each of these num- 
bers? 


a) 01223139784 b) 67021200988 
c) 27041007734 d) 21327903201 


21. One digit in each of these identification numbers of a 
postal money order is smudged. Can you recover the 
smudged digit, indicated by a Q, in each of these num- 
bers? 


a) 49321200688 b) 85009103858 
c) 20941007734 d) 66687003201 

22. Determine which single digit errors are detected by the 
USPS money order code. 


23. Determine which transposition errors are detected by the 
USPS money order code. 


24. Determine the check digit for the UPCs that have these 
initial 11 digits. 
a) 73232184434 b) 63623991346 
c) 04587320720 d) 93764323341 


25. Determine whether each of the strings of 12 digits is a 
valid UPC code. 


a) 036000291452 b) 012345678903 
c) 782421843014 d) 726412175425 

26. Does the check digit of a UPC code detect all single er- 
rors? Prove your answer or find a counterexample. 

27. Determine which transposition errors the check digit of a 
UPC code finds. 


Some airline tickets have a 15-digit identification number 
aja ... d,s, Where a; is a check digit that equals aa, ... a14 
mod 7. 


28. Find the check digit a,; that follows each of these initial 
14 digits of an airline ticket identification number. 
a) 10237424413392 b) 00032781811234 
c) 00611232134231 d) 00193222543435 

29. Determine whether each of these 15-digit numbers is a 
valid airline ticket identification number. 
a) 101333341789013 b) 007862342770445 
c) 113273438882531 d) 000122347322871 


30. Which errors in a single digit of a 15-digit airline ticket 
identification number can be detected? 


*31. Can the accidental transposition of two consecutive dig- 
its in an airline ticket identification number be detected 
using the check digit? 
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Periodicals are identified using an International Standard 
Serial Number (ISSN). An ISSN consists of two blocks of 
four digits. The last digit in the second block is a check 
digit. This check digit is determined by the congruence dg 
= 3d, + 4d, + 5d; + 6d, + 7d; + 8d, + 9d; (mod 11). When 
dg = 10 (mod 11), we use the letter X to represent dą in the 
code. 


32. For each of these initial seven digits of an ISSN, deter- 
mine the check digit (which may be the letter X). 
a) 1570-868 b) 1553-734 
c) 1089-708 d) 1383-811 
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33. 


34. 


35. 


Are each of these eight-digit codes possible ISSNs? That 
is, do they end with a correct check digit? 

a) 1059-1027 b) 0002-9890 

c) 1530-8669 d) 1007-120X 

Does the check digit of an ISSN detect every single error 
in an ISSN? Justify your answer with either a proof or a 
counterexample. 

Does the check digit of an ISSN detect every error where 
two consecutive digits are accidentally interchanged? 
Justify your answer with either a proof or a counter- 
example. 


4.6.1 Introduction 


Number theory plays a key role in cryptography, the subject of transforming information so that 
it cannot be easily recovered without special knowledge. Number theory is the basis of many 
classical ciphers, first used thousands of years ago, and used extensively until the 20th century. 
These ciphers encrypt messages by changing each letter to a different letter, or each block of 
letters to a different block of letters. We will discuss some classical ciphers, including shift 
ciphers, which replace each letter by the letter a fixed number of positions later in the alphabet, 
wrapping around to the beginning of the alphabet when necessary. The classical ciphers we will 
discuss are examples of private key ciphers where knowing how to encrypt allows someone to 
also decrypt messages. With a private key cipher, two parties who wish to communicate in secret 
must share a secret key. The classical ciphers we will discuss are also vulnerable to cryptanalysis, 
which seeks to recover encrypted information without access to the secret information used to 
encrypt the message. We will show how to cryptanalyze messages sent using shift ciphers. 

Number theory is also important in public key cryptography, a type of cryptography in- 
vented in the 1970s. In public key cryptography, knowing how to encrypt does not also tell 
someone how to decrypt. The most widely used public key system, called the RSA cryptosys- 
tem, encrypts messages using modular exponentiation, where the modulus is the product of 
two large primes. Knowing how to encrypt requires that someone know the modulus and an 
exponent. (It does not require that the two prime factors of the modulus be known.) As far as 
it is known, knowing how to decrypt requires someone to know how to invert the encryption 
function, which can only be done in a practical amount of time when someone knows these two 
large prime factors. In this chapter we will explain how the RSA cryptosystem works, including 
how to encrypt and decrypt messages. 

The subject of cryptography also includes the subject of cryptographic protocols, which are 
exchanges of messages carried out by two or more parties to achieve a specific security goal. We 
will discuss two important protocols in this chapter. One allows two people to share a common 
secret key. The other can be used to send signed messages so that a recipient can be sure that 
they were sent by the purported sender. Finally, we will introduce the notion of a homomorphic 
cryptosystem, now playing an important role in cloud computing. Data can become vulnerable 
if they must be decrypted for use as input to programs. Homomorphic encryption eliminates this 
vulnerability by allowing programs to be run on encrypted data. The output of these programs 
is then the encryption of the desired output. 


EXAMPLE 1 


EXAMPLE 2 
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4.6.2 Classical Cryptography 


One of the earliest known uses of cryptography was by Julius Caesar. He made messages secret 
by shifting each letter three letters forward in the alphabet (sending the last three letters of the 
alphabet to the first three). For instance, using this scheme the letter B is sent to E and the letter 
X is sent to A. This is an example of encryption, that is, the process of making a message secret. 

To express Caesar’s encryption process mathematically, first replace each letter by an ele- 
ment of Z», that is, an integer from 0 to 25 equal to one less than its position in the alphabet. For 
example, replace A by 0, K by 10, and Z by 25. Caesar’s encryption method can be represented 
by the function f that assigns to the nonnegative integer p, p < 25, the integer f(p) in the set 
{0, 1, 2, ...,25} with 


f(p) = (p +3) mod 26. 


In the encrypted version of the message, the letter represented by p is replaced with the letter 
represented by (p + 3) mod 26. 


What is the secret message produced from the message “MEET YOU IN THE PARK” using 
the Caesar cipher? 


Solution: First replace the letters in the message with numbers. This produces 
124419 24 14 20 8 13 1974 15017 10. 

Now replace each of these numbers p by f(p) = (p + 3) mod 26. This gives 
157722 11723 11 16 22 107 18 3 20 13. 


Translating this back to letters produces the encrypted message “PHHW BRX LQ WKH 
SDUN.” < 


To recover the original message from a secret message encrypted by the Caesar cipher, the 
function f~', the inverse of f, is used. Note that the function f~! sends an integer p from Z,,, 
to f-!(p) = (p — 3) mod 26. In other words, to find the original message, each letter is shifted 
back three letters in the alphabet, with the first three letters sent to the last three letters of the 
alphabet. The process of determining the original message from the encrypted message is called 
decryption. 

There are various ways to generalize the Caesar cipher. For example, instead of shifting the 
numerical equivalent of each letter by 3, we can shift the numerical equivalent of each letter by 
k, so that 


fp) = (p + k) mod 26. 
Such a cipher is called a shift cipher. Note that decryption can be carried out using 
f~! (p) = (p — k) mod 26. 


Here the integer k is called a key. We illustrate the use of a shift cipher in Examples 2 and 3. 


Encrypt the plaintext message “STOP GLOBAL WARMING” using the shift cipher with shift 
k=11. 
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EXAMPLE 3 


EXAMPLE 4 


Solution: To encrypt the message “STOP GLOBAL WARMING” we first translate each letter 
to the corresponding element of Z,,. This produces the string 


1819 1415 611141011 22017128 136. 
We now apply the shift f(p) = (p + 11) mod 26 to each number in this string. We obtain 
34250 17 22 25 12 11 22 711223 19 24 17. 


Translating this last string back to letters, we obtain the ciphertext “DEZA RWZMLW HLCX- 
TYR.” < 


Decrypt the ciphertext message “LEWLYPLUJL PZ H NYLHA ALHJOLY” that was en- 
crypted with the shift cipher with shift k = 7. 


Solution: To decrypt the ciphertext “LEWLYPLUJL PZ H NYLHA ALHJOLY” we first 
translate the letters back to elements of Z3. We obtain 


1142211 24151120911 15 25 7 13241170 011791411 24. 
Next, we shift each of these numbers by —k = —7 modulo 26 to obtain 
423 15417841324 8 18 0 6174019 194027417. 


Finally, we translate these numbers back to letters to obtain the plaintext. We obtain 
“EXPERIENCE IS A GREAT TEACHER.” < 


We can generalize shift ciphers further to slightly enhance security by using a function of 
the form 


JI (p) = (ap + b) mod 26, 


where a and b are integers, chosen so that f is a bijection. (The function f (p) = (ap + b) mod 26 
is a bijection if and only if gcd(a, 26) = 1.) Such a mapping is called an affine transformation, 
and the resulting cipher is called an affine cipher. 


What letter replaces the letter K when the function f(p) = (7p +3) mod 26 is used for 
encryption? 


Solution: First, note that 10 represents K. Then, using the encryption function specified, it fol- 
lows that f(10) = (7 - 10 + 3) mod 26 = 21. Because 21 represents V, K is replaced by V in the 
encrypted message. < 


We will now show how to decrypt messages encrypted using an affine cipher. Suppose 
that c = (ap + b) mod 26 with gcd(a, 26) = 1. To decrypt we need to show how to express p in 
terms of c. To do this, we apply the encrypting congruence c = ap + b(mod 26), and solve it 
for p. To do this, we first subtract b from both sides, to obtain c — b = ap (mod 26). Because 
gcd(a, 26) = 1, we know that there is an inverse a of a modulo 26. Multiplying both sides of 
the last equation by a gives us a(c — b) = aap (mod 26). Because aa = 1 (mod 26), this tells 
us that p = a(c — b) (mod 26). This determines p because p belongs to Z,- 


CRYPTANALYSIS The process of recovering plaintext from ciphertext without knowledge 
of both the encryption method and the key is known as cryptanalysis or breaking codes. In 
general, cryptanalysis is a difficult process, especially when the encryption method is unknown. 


Mathematicians make 
the best code breakers. 
Their work in World 
War II changed the 
course of the war. 


EXAMPLE 5 


Links > 


EXAMPLE 6 


4.6 Cryptography 313 


We will not discuss cryptanalysis in general, but we will explain how to break messages that 
were encrypted using a shift cipher. 

If we know that a ciphertext message was produced by enciphering a message using a shift 
cipher, we can try to recover the message by shifting all characters of the ciphertext by each 
of the 26 possible shifts (including a shift of zero characters). One of these is guaranteed to be 
the plaintext. However, we can use a more intelligent approach, which we can build upon to 
cryptanalyze ciphertext resulting from other ciphers. The main tool for cryptanalyzing cipher- 
text encrypted using a shift cipher is the count of the frequency of letters in the ciphertext. The 
nine most common letters in English text and their approximate relative frequencies are E 13%, 
T9%, A 8%, O 8%, 17%, N 7%, S 7%, H 6%, and R 6%. To cryptanalyze ciphertext that we know 
was produced using a shift cipher, we first find the relative frequencies of letters in the cipher- 
text. We list the most common letters in the ciphertext in frequency order; we hypothesize that 
the most common letter in the ciphertext is produced by encrypting E. Then, we determine the 
value of the shift under this hypothesis, say k. If the message produced by shifting the ciphertext 
by —k makes sense, we presume that our hypothesis is correct and that we have the correct value 
of k. If it does not make sense, we next consider the hypothesis that the most common letter in 
the ciphertext is produced by encrypting T, the second most common letter in English; we find k 
under this hypothesis, shift the letters of the message by —k, and see whether the resulting mes- 
sage makes sense. If it does not, we continue the process working our way through the letters 
from most common to least common. 


Suppose that we intercepted the ciphertext message ZNK KGXRE HOXJ MKZY ZNK CUXS 
that we know was produced by a shift cipher. What was the original plaintext message? 


Solution: Because we know that the intercepted ciphertext message was encrypted using a 
shift cipher, we begin by calculating the frequency of letters in the ciphertext. We find that 
the most common letter in the ciphertext is K. So, we hypothesize that the shift cipher sent 
the plaintext letter E to the ciphertext letter K. If this hypothesis is correct, we know that 
10=4+k mod 26, so k= 6. Next, we shift the letters of the message by —6, obtaining 
THE EARLY BIRD GETS THE WORM. Because this message makes sense, we assume that 
the hypothesis that k = 6 is correct. < 


BLOCK CIPHERS Shift ciphers and affine ciphers proceed by replacing each letter of the 
alphabet by another letter in the alphabet. Because of this, these ciphers are called character or 
monoalphabetic ciphers. Encryption methods of this kind are vulnerable to attacks based on 
the analysis of letter frequency in the ciphertext, as we just illustrated. We can make it harder 
to successfully attack ciphertext by replacing blocks of letters with other blocks of letters in- 
stead of replacing individual characters with individual characters; such ciphers are called block 
ciphers. 

We will now introduce a simple type of block cipher, called the transposition cipher. As 
a key we use a permutation o of the set {1, 2, ..., m} for some positive integer m, that is, a 
one-to-one function from {1, 2, ..., m} to itself. To encrypt a message we first split its letters 
into blocks of size m. (If the number of letters in the message is not divisible by m we add 
some random letters at the end to fill out the final block.) We encrypt the block p;p ...p,, as 
C1 Co +++ Cm =Po) Po) +++» Pom TO decrypt a ciphertext block c;c ... Cn, we transpose its letters 
using the permutation o~!, the inverse of o. Example 6 illustrates encryption and decryption 
for a transposition cipher. 


Using the transposition cipher based on the permutation o of the set {1, 2, 3,4} with o(1) = 3, 
o(2) = 1, o(3) = 4, and o(4) = 2, 


(a) Encrypt the plaintext message PIRATE ATTACK. 
(b) Decrypt the ciphertext message SWUE TRAE OEHS, which was encrypted using this cipher. 
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Solution: (a) We first split the letters of the plaintext into blocks of four letters. We obtain PIRA 
TEAT TACK. To encrypt each block, we send the first letter to the third position, the second 
letter to the first position, the third letter to the fourth position, and the fourth letter to the second 
position. We obtain IAPR ETTA AKTC. 

(b) We note that o~!, the inverse of o, sends 1 to 2, sends 2 to 4, sends 3 to 1, and sends 
4 to 3. Applying o~!(m) to each block gives us the plaintext: USEW ATER HOSE. (Grouping 
together these letters to form common words, we surmise that the plaintext is USE WATER 
HOSE.) < 


CRYPTOSYSTEMS We have defined two families of ciphers: shift ciphers and affine ciphers. 
We now introduce the notion of a cryptosystem, which provides a general structure for defining 
new families of ciphers. 


A cryptosystem is a five-tuple (P, C, K, €, D), where P is the set of plaintext strings, C is the 
set of ciphertext strings, K is the keyspace (the set of all possible keys), € is the set of en- 
cryption functions, and D is the set of decryption functions. We denote by £, the encryption 
function in € corresponding to the key k and D, the decryption function in D that decrypts 
ciphertext that was encrypted using E,, that is, D,(E,(p)) = p, for all plaintext strings p. 


We now illustrate the use of the definition of a cryptosystem. 


Describe the family of shift ciphers as a cryptosytem. 


Solution: To encrypt a string of English letters with a shift cipher, we first translate each letter 
to an integer between 0 and 25, that is, to an element of Z6. We then shift each of these integers 
by a fixed integer modulo 26, and finally, we translate the integers back to letters. To apply the 
definition of a cryptosystem to shift ciphers, we assume that our messages are already integers, 
that is, elements of Z,,. That is, we assume that the translation between letters and integers is 
outside of the cryptosystem. Consequently, both the set of plaintext strings P and the set of 
ciphertext strings C are the set of strings of elements of Z,¢. The set of keys XK is the set of 
possible shifts, so K = Z,.. The set € consists of functions of the form E,(p) = (p + k) mod 26, 
and the set D of decryption functions is the same as the set of encrypting functions where 
D,(p) = (p — k)mod 26. < 


The concept of a cryptosystem is useful in the discussion of additional families of ciphers 
and is used extensively in cryptography. 


4.6.3 Public Key Cryptography 


All classical ciphers, including shift ciphers and affine ciphers, are examples of private key 
cryptosystems. In a private key cryptosystem, once you know an encryption key, you can 
quickly find the decryption key. So, knowing how to encrypt messages using a particular key 
allows you to decrypt messages that were encrypted using this key. For example, when a shift 
cipher is used with encryption key k, the plaintext integer p is sent to 


c = (p +k) mod 26. 


Decryption is carried out by shifting by —k; that is, 
p = (c — k) mod 26. 


So knowing how to encrypt with a shift cipher also tells you how to decrypt. 


M.LT. is also known as 
the *Tute. 


Unfortunately, no one 
calls this the Cocks 
cryptosystem. 
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When a private key cryptosystem is used, two parties who wish to communicate in secret 
must share a secret key. Because anyone who knows this key can both encrypt and decrypt mes- 
sages, two people who want to communicate securely need to securely exchange this key. (We 
will introduce a method for doing this later in this section.) The shift cipher and affine cipher 
cryptosystems are private key cryptosystems. They are quite simple and are extremely vulnera- 
ble to cryptanalysis. However, the same is not true of many modern private key cryptosystems. 
In particular, the current US government standard for private key cryptography, the Advanced 
Encryption Standard (AES), is extremely complex and is considered to be highly resistant to 
cryptanalysis. (See [St06] for details on AES and other modern private key cryptosystems.) 
AES is widely used in government and commercial communications. However, it still shares 
the property that for secure communications keys be shared. Furthermore, for extra security, a 
new key is used for each communication session between two parties, which requires a method 
for generating keys and securely sharing them. 

To avoid the need for keys to be shared by every pair of parties that wish to communi- 
cate securely, in the 1970s cryptologists introduced the concept of public key cryptosystems. 
When such cryptosystems are used, knowing how to send an encrypted message does not help 
decrypt messages. In such a system, everyone can have a publicly known encryption key. Only 
the decryption keys are kept secret, and only the intended recipient of a message can decrypt 
it, because, as far as it is currently known, knowledge of the encryption key does not let some- 
one recover the plaintext message without an extraordinary amount of work (such as billions of 
years of computer time). 

The first public key cryptosystems were invented in the mid-1970s. Many additional public 
key cryptosystems have been developed in the ensuing decades. We will introduce the most 
commonly used public key cryptosystem, known as the RSA system, in this book. Besides RSA, 
there are several other commonly used public key cryptosystems that are now used for many 
applications. These other public key cryptosystems will play an important role in the future 
when advances in commuting may make the RSA cryptosystem obsolete, as often happens in 
cryptography. We will explain why this may be so shortly. 

Although public key cryptography has the advantage that two parties who wish to com- 
municate securely do not need to exchange keys, it has the disadvantage that encryption and 
decryption can be extremely time-consuming. For many applications, this makes public key 
cryptography impractical. In such situations, private key cryptography is used instead. How- 
ever, public key cryptography may still be used in the key exchange process. 


4.6.4 The RSA Cryptosystem 


In 1976, three researchers at the Massachusetts Institute of Technology—Ronald Rivest, Adi 
Shamir, and Leonard Adleman—introduced to the world a public key cryptosystem, known as 
the RSA system, from the initials of its inventors. As often happens with cryptographic discov- 
eries, the RSA system had been discovered several years earlier in secret government research in 
the United Kingdom. Clifford Cocks, working in secrecy at the United Kingdom’s Government 
Communications Headquarters (GCHQ), had discovered this cryptosystem in 1973. However, 
his invention was unknown to the outside world until the late 1990s, when he was allowed to 
share classified GCHQ documents from the early 1970s. (An excellent account of this earlier 
discovery, as well as the work of Rivest, Shamir, and Adleman, can be found in [Si99].) 

In the RSA cryptosystem, each individual has an encryption key (n, e), where n = pq, the 
modulus is the product of two large primes p and q, say with 300 digits each, and an exponent 
e that is relatively prime to (p — 1)(g — 1). To produce a usable key, two large primes must be 
found. This can be done quickly on a computer using probabilistic primality tests, referred to 
earlier in this section. However, the product of these primes n = pq, with approximately 600 
digits, cannot, as far as is currently known, be factored in a reasonable length of time. As we 
will see, this is an important reason why decryption cannot, as far as is currently known, be 
done quickly without a separate decryption key. 
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Remark: With the steady increase of the speed of computers, the recommended size of the 
primes p and q used to produce a RSA public key has increased. But the larger n is, the slower 
RSA encryption and decryption become. When considering this tradeoff, the number of years 
a message needs to be remain secret is important. A more important consideration is that the 
development of quantum computing threatens the security of the RSA cryptosystem, because 
factorization algorithms have been developed for quantum computers that could then be used to 
quickly factor large primes. So, once quantum computing becomes practical, perhaps in the next 
20 to 30 years, other public key cryptosystems that cannot be broken using quantum computing 
will need to be used. 


4.6.5 RSA Encryption 


To encrypt messages using a particular key (n, e), we first translate a plaintext message M 
into sequences of integers. To do this, we first translate each plaintext letter into a two-digit 
number, using the same translation we employed for shift ciphers, with one key difference. 
That is, we include an initial zero for the letters A through J, so that A is translated into 00, 
B into 01, ... , and J into 09. Then, we concatenate these two-digit numbers into strings of dig- 
its. Next, we divide this string into equally sized blocks of 2N digits, where 2N is the largest 
even number such that the number 2525 ... 25 with 2N digits does not exceed n. (When neces- 
sary, we pad the plaintext message with dummy Xs to make the last block the same size as all 
other blocks.) 

After these steps, we have translated the plaintext message M into a sequence of integers 
mı, My, ...,m, for some integer k. Encryption proceeds by transforming each block m, to a 
ciphertext block c;. This is done using the function 


c = m° mod n. 


(To perform the encryption, we use an algorithm for fast modular exponentiation, such as Algo- 
rithm 5 in Section 4.2.) We leave the encrypted message as blocks of numbers and send these to 
the intended recipient. Because the RSA cryptosystem encrypts blocks of characters into blocks 
of characters, it is a block cipher. 

Example 8 illustrates how RSA encryption is performed. For practical reasons we use small 
primes p and q in this example, rather than primes with 300 or more digits. Although the cipher 
described in this example is not secure, it does illustrate the techniques used in the RSA cipher. 


Encrypt the message STOP using the RSA cryptosystem with key (2537, 13). Note that 2537 = 
43 - 59, p = 43 and q = 59 are primes, and 


gcd(e, (p — 1)(q — 1)) = ged(13, 42 - 58) = 1. 


CLIFFORD COCKS (BORN 1950) Clifford Cocks, born in Cheshire, England, was a talented mathe- 
matics student. He attended the Manchester Grammar School. In 1968 he won a silver medal at the In- 
ternational Mathematical Olympiad. Cocks attended King’s College, Cambridge, studying mathematics. He 
also spent a short time at Oxford University working in number theory. In 1973 he decided not to com- 
plete his graduate work, instead taking a mathematical job at the Government Communications Headquarters 
(GCHQ) of British intelligence. Two months after joining GCHQ, Cocks learned about public key cryptog- 
raphy from an internal GCHQ report written by James Ellis. Cocks used his number theory knowledge to 
invent what is now called the RSA cryptosystem. He quickly realized that a public key cryptosystem could 
be based on the difficulty of reversing the process of multiplying two large primes. In 1997 he was allowed 
to reveal declassified GCHQ internal documents describing his discovery. Cocks is also known for his in- 
vention of a secure identity-based encryption scheme, which uses information about a user’s identity as a 


public key. In 2001, Cocks became the Chief Mathematician at GCHQ. He has also set up the Heilbronn Institute for Mathematical 
Research, a partnership between GCHQ and the University of Bristol. 
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Solution: To encrypt, we first translate the letters in STOP into their numerical equivalents. We 
then group these numbers into blocks of four digits (because 2525 < 2537 < 252525), to obtain 


1819 1415. 


We encrypt each block using the mapping 
c = m”? mod 2537. 


Computations using fast modular multiplication show that 1819! mod 2537 = 2081 and 
1415! mod 2537 = 2182. The encrypted message is 2081 2182. < 


4.6.6 RSA Decryption 


The plaintext message can be quickly recovered from a ciphertext message when the decryp- 
tion key d, an inverse of e modulo (p — 1)(q — 1), is known. [Such an inverse exists because 
gcd(e, (p — 1)(q — 1)) = 1.] To see this, note that if de = 1 (mod (p — 1)(q — 1)), there is an 
integer k such that de = 1 + k(p — 1)(q — 1). It follows that 


cf = (mA! = m® = m'tke-YG-Y (mod n). 


By Fermat’s little theorem [assuming that gcd(m, p) = gcd(m, q) = 1, which holds except in rare 
cases, which we cover in Exercise 28], it follows that m?-! = 1 (mod p) and m?~! = 1 (mod q). 
Consequently, 


cf = m- (m?"!)K-) = m- 1 = m (mod p) 


RONALD RIVEST (BORN 1948) Ronald Rivest received a B.A. from Yale in 1969 and his Ph.D. in computer 
science from Stanford in 1974. Rivest is a computer science professor at M.I.T. and was a cofounder of RSA Data 
Security, which held the patent on the RSA cryptosystem that he invented together with Adi Shamir and Leonard 
Adleman. Areas that Rivest has worked in besides cryptography include machine learning, VLSI design, and 
computer algorithms. He is a coauthor of a popular text on algorithms ([CoLeRiSt09]). 


ADI SHAMIR (BORN 1952) Adi Shamir was born in Tel Aviv, Israel. His undergraduate degree is from 
Tel Aviv University (1972) and his Ph.D. is from the Weizmann Institute of Science (1977). Shamir was a 
research assistant at the University of Warwick and an assistant professor at M.I.T. He is currently a profes- 
sor in the Applied Mathematics Department at the Weizmann Institute and leads a group studying computer 
security. Shamir’s contributions to cryptography, besides the RSA cryptosystem, include cracking knapsack 
cryptosystems, cryptanalysis of the Data Encryption Standard (DES), and the design of many cryptographic 
protocols. 


LEONARD ADLEMAN (BORN 1945) Leonard Adleman was born in San Francisco, California. He received 
a B.S. in mathematics (1968) and his Ph.D. in computer science (1976) from the University of California, 
Berkeley. Adleman was a member of the mathematics faculty at M.L.T. from 1976 until 1980, where he was 
a coinventor of the RSA cryptosystem, and in 1980 he took a position in the computer science department at 
the University of Southern California (USC). He was appointed to a chaired position at USC in 1985. Adleman 
has worked on computer security, computational complexity, immunology, and molecular biology. He invented 
the term “computer virus.” Adleman’s recent work on DNA computing has sparked great interest. He was a 
technical adviser for the movie Sneakers, in which computer security played an important role. 
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and 
c! =m- (mT!) = m. 1 =m (mod qg): 

Because gcd(p, q) = 1, it follows by the Chinese remainder theorem that 
c! = m (mod pq). 


Example 9 illustrates how to decrypt messages sent using the RSA cryptosystem. 


We receive the encrypted message 0981 0461. What is the decrypted message if it was encrypted 
using the RSA cipher from Example 8? 


Solution: The message was encrypted using the RSA cryptosystem with n = 43 - 59 and expo- 
nent 13. As Exercise 2 in Section 4.4 shows, d = 937 is an inverse of 13 modulo 42 - 58 = 2436. 
We use 937 as our decryption exponent. Consequently, to decrypt a block c, we compute 


m = cc?! mod 2537. 


To decrypt the message, we use the fast modular exponentiation algorithm to compute 
0981°°7 mod 2537 = 0704 and 0461977 mod 2537 = 1115. Consequently, the numerical version 
of the original message is 0704 1115. Translating this back to English letters, we see that the 
message is HELP. < 


4.6.7 RSA as a Public Key System 


Why is the RSA cryptosystem suitable for public key cryptography? First, itis possible to rapidly 
construct a public key by finding two large primes p and g, each with more than 300 digits, 
and to find an integer e relatively prime to (p — 1)(q — 1). When we know the factorization of 
the modulus n, that is, when we know p and q, we can quickly find an inverse d of e modulo 
(p — 1)(q — 1). [This is done by using the Euclidean algorithm to find Bézout coefficients s 
and ¢ for d and (p — 1)(q — 1), which shows that the inverse of d modulo (p — 1)(q — 1) is s 
mod (p — 1)(q — 1).] Knowing d lets us decrypt messages sent using our key. However, no 
method is known to decrypt messages that is not based on finding a factorization of n, or that 
does not also lead to the factorization of n. 

Factorization is believed to be a difficult problem, as opposed to finding large primes p 
and q, which can be done quickly. The most efficient factorization methods known (as of 2017) 
require billions of years to factor 600-digit integers. Consequently, when p and q are 300-digit 
primes, it is believed that messages encrypted using n = pq as the modulus cannot be decrypted 
in a reasonable time unless the primes p and q are known. 

Although no polynomial-time algorithm is known for factoring large integers, active re- 
search is under way to find new ways to efficiently factor integers. Integers that were thought, 
as recently as several years ago, to be far too large to be factored in a reasonable amount of time 
can now be factored routinely. Integers with more than 230 digits have been factored using team 
efforts. When new factorization techniques are found, it will be necessary to use larger primes 
to ensure the secrecy of messages. Unfortunately, messages that were considered secure earlier 
can be saved and subsequently decrypted by unintended recipients when it becomes feasible 
to factor the n = pq in the key used for RSA encryption. (Note that the RSA system will be 
insecure once quantum computing is available.) 

The RSA method is now widely used. However, the most commonly used cryptosystems 
are private key cryptosystems. The use of public key cryptography, via the RSA system, is 
growing. Nevertheless, there are applications that use both private key and public key systems. 
For example, a public key cryptosystem, such as RSA, can be used to distribute private keys 
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to pairs of individuals when they wish to communicate. These people then use a private key 
system for the encryption and decryption of messages. 


4.6.8 Cryptographic Protocols 


So far we have shown how cryptography can be used to make messages secure. However, there 
are many other important applications of cryptography. Among these applications are cryp- 
tographic protocols, which are exchanges of messages carried out by two or more parties to 
achieve a particular security goal. In particular, we will show how cryptography can be used to 
allow two people to exchange a secret key over an insecure communication channel. We will 
also show how cryptography can be used to send signed secret messages so that the recipient 
can be sure that the message came from the purported sender. We refer the reader to [St05] for 
thorough discussions of a variety of cryptographic protocols. 


KEY EXCHANGE We now discuss a protocol that two parties can use to exchange a secret 
key over an insecure communications channel without having shared any information in the 
past. Generating a key that two parties can share is important for many applications of cryp- 
tography. For example, for two people to send secure messages to each other using a private 
key cryptosystem they need to share a common key. The protocol we will describe is known as 
the Diffie-Hellman key agreement protocol, after Whitfield Diffie and Martin Hellman, who 
described it in 1976. However, this protocol was invented in 1974 by Malcolm Williamson 
in secret work at the British GCHQ. It was not until 1997 that his discovery was made 
public. 


Suppose that Alice and Bob want to share a common key. The protocol follows these steps, 
where the computations are done in Z,,. 


(1) Alice and Bob agree to use a prime p and a primitive root a of p. 
(2) Alice chooses a secret integer kų and sends a“ mod p to Bob. 

(3) Bob chooses a secret integer k, and sends a% mod p to Alice. 

(4) Alice computes (a2) mod p. 

(5) Bob computes (a )© mod p. 


At the end of this protocol, Alice and Bob have computed their shared key, namely, 
(a2) mod p = (ak). mod p. 


To analyze the security of this protocol, note that the messages sent in steps (1), (2), and 
(3) are not assumed to be sent securely. We can even assume that these communications were 
in the clear and that their contents are public information. So, p, a, a“ mod p, and a% mod p 
are assumed to be public information. The protocol ensures that k,, k), and the common key 
(a2) mod p = (a) mod p are kept secret. To find the secret information from this public 
information requires that an adversary solves instances of the discrete logarithm problem, be- 
cause the adversary would need to find k, and k, from a" mod p and a mod p, respectively. 
Furthermore, no other method is known for finding the shared key using just the public infor- 
mation. We have remarked that this is thought to be computationally infeasible when p and a 
are sufficiently large. With the computing power available now, this system is considered un- 
breakable when p has more than 300 decimal digits and k, and k, have more than 100 decimal 
digits each. 
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DIGITAL SIGNATURES Not only can cryptography be used to secure the confidentiality of 
a message, but it also can be used so that the recipient of the message knows that it came from 
the person they think it came from. We first show how a message can be sent so that a recipient 
of the message will be sure that the message came from the purported sender of the message. 
In particular, we can show how this can be accomplished using the RSA cryptosystem to apply 
a digital signature to a message. 

Suppose that Alice’s RSA public key is (n, e) and her private key is d. Alice encrypts a 
plaintext message x using the encryption function E; .(x) = x° mod n. She decrypts a ciphertext 
message y using the decryption function Doe) = x4 mod n. Alice wants to send the message 
M so that everyone who receives the message knows that it came from her. Just as in RSA 
encryption, she translates the letters into their numerical equivalents and splits the resulting 
string into blocks m, mp, ...,m, such that each block is the same size, which is as large as 
possible so that 0 < m; < n fori = 1, 2, ..., k. She then applies her decryption function Dine) to 
each block, obtaining D,, ,(m,), i= 1, 2, ..., k. She sends the result to all intended recipients of 
the message. 

When a recipient receives her message, they apply Alice’s encryption function Eg, e) to each 
block, which everyone has available because Alice’s key (n, e) is public information. The result 
is the original plaintext block because E (Dine) = x. So, Alice can send her message to as 
many people as she wants and by signing it in this way, every recipient can be sure it came from 
Alice. Example 10 illustrates this protocol. 


n,e 


Suppose Alice’s public RSA cryptosystem key is the same as in Example 8. That is, n = 43 - 
59 = 2537 and e = 13. Her decryption key is d = 937, as described in Example 9. She wants 
to send the message “MEET AT NOON” to her friends so that they are sure it came from her. 
What should she send? 


Solution: Alice first translates the message into blocks of digits, obtaining 1204 0419 0019 1314 
1413 (as the reader should verify). She then applies her decryption transformation D(9537,13)(x) = 
x37 mod 2537 to each block. Using fast modular exponentiation (with the help of a computa- 
tional aid), she finds that 1204°7 mod 2537 = 817, 419°°’ mod 2537 = 555, 19°” mod 2537 = 
1310, 1314°°7 mod 2537 = 2173, and 141357 mod 2537 = 1026. 

So, the message she sends, split into blocks, is 0817 0555 1310 2173 1026. When one of 
her friends gets this message, they apply her encryption transformation E(9537 13) to each block. 
When they do this, they obtain the blocks of digits of the original message, which they translate 
back to English letters. < 


We have shown that signed messages can be sent using the RSA cryptosystem. We can 
extend this by sending signed secret messages. To do this, the sender applies RSA encryption 
using the publicly known encryption key of an intended recipient to each block that was en- 
crypted using the sender’s decryption transformation. The recipient then first applies his private 
decryption transformation and then the sender’s public encryption transformation. (Exercise 32 
asks for this protocol to be carried out.) 


4.6.9 Homomorphic Encryption 


A cryptosystem, such as RSA, can be used to encrypt files to keep them secret. Today, many 
users store encrypted files in the cloud, where they reside on remote computers. It is often 
necessary to run programs using data from files stored on the cloud. These data are vulnerable 
to users with access to the remote computer where our data are stored if we run programs on the 
cloud without downloading these data. These data are also vulnerable to eavesdroppers when 
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we download files, run programs on our computer, and upload the output to the cloud. Could we 
avoid these vulnerabilities by just running programs on encrypted data? Although this seems 
farfetched at first blush, in 1979, soon after RSA was introduced, the question was proposed 
whether there was a cryptosystem that allowed arbitrary computations to be run on encrypted 
data that would produce the encryption of the unencrypted output produced by this unencrypted 
input. For such a cryptosystem, it would not be necessary to decrypt input data because the 
program could be run on a remote system without disclosing either the input or the output. So, 
the search began for a fully homomorphic cryptosystem that allows arbitrary computations to 
be run remotely on encrypted data. 

Before we discuss progress towards fully homomorphic encryption, we will show that the 
RSA cryptosytem is not fully homomorphic, although it allows some computations to be done 
on encrypted data. 


RSA is Partially Homomorphic Let (n, e) be a public key for the RSA cryptosystem and sup- 
pose that M, and M, are plaintext messages, so that 0 < M, < n and 0 < M, <n. Then 


E e)M)E(n,e(M2) mod m = (M{ mod m - M; mod m) mod m 
= (M,M,)° mod m= E(n,e)(M M2). 


From this equation we see that Ey, (M1) ‘n Eno (M2) = En, (M, M2) in Z,. Because of 
this, we say that RSA is multiplicatively homomorphic. When we have encrypted using RSA, 
we can carry out multiplications without first decrypting because the encryption of the product 
of two plaintexts equals the product of their encryptions. 

However, it is not true that Ep (M1) +, E(,.)(M2) = Ege) (M, + M3) for all M, and M, in 
Z,,. (For instance, this is easy to see when M, = 1.) That is, when we have encrypted with RSA, 
we cannot add the encryptions of two numbers to obtain the encryption of their sum. More- 
over, there is no method known for determining E, (M1 + M2) from Eg .(M1) and Ep .(M2) 
without decrypting these two ciphertexts. We say that RSA is not additively homomorphic. 
Because it is multiplicatively homomorphic, but not additively homomorphic, RSA is partially 
homomorphic. < 


In 2009, Craig Gentry described the first fully homomorphic cryptosystem, based on what 
is known as lattice-based cryptography. Unfortunately, no practical fully homomorphic cryp- 
tosystems have yet been developed, because all require extremely large amounts of processing 
and memory. However, there is hope that new advances will lead to practical fully homomorphic 
cryptosystems in the not so distant future. 


CRAIG B. GENTRY (BORN 1972) Craig B. Gentry received his B.S. from Duke University in 1993 and 
his J.D. from Harvard University Law School in 1998. For two years he worked as an intellectual property 
lawyer. From 2000 to 2005 he was a senior research engineer at NTT DoCoMo USA Labs. He decided to return 
to school, and in 2009 he received a Ph.D. in computer science from Stanford University. Gentry joined the 
Cryptography Research Group of the IBM Watson Research Center in 2009 where he currently works. 

Gentry’s invention of a fully homomorphic scheme, which resolved an open problem proposed in 1978, 
earned him the 2010 ACM Grace Murray Hopper Award. In 2013, Gentry with others constructed the first cryp- 
tographic multilinear maps, using them to construct the first cryptographic program obfuscation schemes, which 
many had thought might not exist. Gentry’s work on fully homomorphic cryptography and on cryptographic 
multilinear maps is based on lattice-based cryptography, which cannot be broken by quantum computing, unlike 
the RSA cryptosystem. Gentry, along with his colleagues, advanced the area of verifiable computation, which 
allows a computer to offload the computation of functions to other computers while maintaining verifiable 
results. In 2014, Gentry was awarded a MacArthur Fellowship (also known as a Genuis Grant). 
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Exercises 


1. Encrypt the message DO NOT PASS GO by translat- 
ing the letters into numbers, applying the given encryp- 
tion function, and then translating the numbers back into 
letters. 


a) f(p) = (p + 3) mod 26 (the Caesar cipher) 
b) f(y) = (p + 13) mod 26 
c) f(y) = Bp +7) mod 26 


2. Encrypt the message STOP POLLUTION by translat- 
ing the letters into numbers, applying the given encryp- 
tion function, and then translating the numbers back into 
letters. 


a) f(p) = (p + 4) mod 26 
b) f(p) = (p + 21) mod 26 
c) f(p) = (7p + 22) mod 26 


3. Encrypt the message WATCH YOUR STEP by translat- 
ing the letters into numbers, applying the given encryp- 
tion function, and then translating the numbers back into 
letters. 


a) f(p) = (p + 14) mod 26 
b) f(p) = (14p + 21) mod 26 
c) f(p) = (—7p + 1) mod 26 


4. Decrypt these messages that were encrypted using the 
Caesar cipher. 


a) EOXH MHDQV 
b) WHVW WRGDB 
c) HDW GLP VXP 


5. Decrypt these messages encrypted using the shift cipher 
fp) = (p + 10) mod 26. 


a) CEBBOXNOB XYG 
b) LO WI PBSOXN 
c) DSWO PYB PEX 


6. Suppose that when a long string of text is encrypted using 
a shift cipher f(p) = (p + k) mod 26, the most common 
letter in the ciphertext is X. What is the most likely value 
for k, assuming that the distribution of letters in the text 
is typical of English text? 


7. Suppose that when a string of English text is encrypted 
using a shift cipher f(p) = (p + k) mod 26, the resulting 
ciphertext is DY CVOOZ ZOBMRKXMO DY NBOKW. 
What was the original plaintext string? 


8. Suppose that the ciphertext DVE CFMV KF NFEUVI, 
REU KYRK ZJ KYV JVVU FW JTZVETV was pro- 
duced by encrypting a plaintext message using a shift ci- 
pher. What is the original plaintext? 


9. Suppose that the ciphertext ERC WYJJMGMIRXPC 
EHZERGIH XIGLRSPSKC MW MRHMWXM- 
RKYMWLEFPI JVSQ QEKMG was produced by en- 
crypting a plaintext message using a shift cipher. What 
is the original plaintext? 


10. Determine whether there is a key for which the encipher- 
ing function for the shift cipher is the same as the deci- 
phering function. 

11. What is the decryption function for an affine cipher if the 
encryption function is c = (15p + 13) mod 26? 

* 12. Find all pairs of integers keys (a, b) for affine ciphers for 
which the encryption function c = (ap + b) mod 26 is the 
same as the corresponding decryption function. 

13. Suppose that the most common letter and the second 
most common letter in a long ciphertext produced by en- 
crypting a plaintext using an affine cipher f(p) = (ap + 
b) mod 26 are Z and J, respectively. What are the most 
likely values of a and b? 

14. Encrypt the message GRIZZLY BEARS using blocks 
of five letters and the transposition cipher based on the 
permutation of {1, 2,3,4,5} with o(1) = 3, o(2) = 5, 
o(3) = 1, o(4) = 2, and o(5) = 4. For this exercise, use 
the letter X as many times as necessary to fill out the final 
block of fewer then five letters. 

15. Decrypt the message EABW EFRO ATMR ASIN, which 
is the ciphertext produced by encrypting a plaintext mes- 
sage using the transposition cipher with blocks of four 
letters and the permutation o of {1, 2, 3,4} defined by 
o(1) = 3, o(2) = 1, o(3) = 4, and o(4) = 2. 

*16. Suppose that you know that a ciphertext was produced 
by encrypting a plaintext message with a transposition 
cipher. How might you go about breaking it? 

17. Suppose you have intercepted a ciphertext message and 
when you determine the frequencies of letters in this mes- 
sage, you find the frequencies are similar to the frequency 
of letters in English text. Which type of cipher do you 
suspect was used? 


The Vigenére cipher is a block cipher, with a key that is a 
string of letters with numerical equivalents k,k, ...k,,, where 
k; E Z fori = 1, 2, ..., m. Suppose that the numerical equiv- 
alents of the letters of a plaintext block are p;p, ... Pm- The 
corresponding numerical ciphertext block is (p, + k,) mod 26 
(P2 +k) mod 26... (Pm + kn) mod 26. Finally, we translate 
back to letters. For example, suppose that the key string is 
RED, with numerical equivalents 17 4 3. Then, the plaintext 
ORANGE, with numerical equivalents 14 17 00 13 06 04, is 
encrypted by first splitting it into two blocks 14 17 00 and 13 
06 04. Then, in each block we shift the first letter by 17, the 
second by 4, and the third by 3. We obtain 5 21 03 and 04 10 
07. The cipherext is FVDEKH. 
18. Use the Vigenére cipher with key BLUE to encrypt the 
message SNOWFALL. 
19. The ciphertext OIK YW VHBX was produced by encrypt- 
ing a plaintext message using the Vigenère cipher with 
key HOT. What is the plaintext message? 


20. Express the Vigenère cipher as a cryptosystem. 
To break a Vigenére cipher by recovering a plaintext message 


from the ciphertext message without having the key, the first 
step is to figure out the length of the key string. The second 


step is to figure out each character of the key string by deter- 

mining the corresponding shift. Exercises 21 and 22 deal with 

these two aspects. 

21. Suppose that when a long string of text is encrypted using 
a Vigenère cipher, the same string is found in the cipher- 
text starting at several different positions. Explain how 
this information can be used to help determine the length 
of the key. 

22. Once the length of the key string of a Vigénere cipher is 
known, explain how to determine each of its characters. 
Assume that the plaintext is long enough so that the fre- 
quency of its letters is reasonably close to the frequency 
of letters in typical English text. 

*23. Show that we can easily factor n when we know that n 
is the product of two primes, p and q, and we know the 
value of (p — 1)(q — 1). 

In Exercises 24—27 first express your answers without com- 

puting modular exponentiations. Then use a computational 

aid to complete these computations. 

24. Encrypt the message ATTACK using the RSA system 
with n = 43 - 59 and e = 13, translating each letter into 
integers and grouping together pairs of integers, as done 
in Example 8. 

25. Encrypt the message UPLOAD using the RSA system 
with n = 53 - 61 and e = 17, translating each letter into 
integers and grouping together pairs of integers, as done 
in Example 8. 

26. What is the original message encrypted using the RSA 
system with n = 53 - 61 and e = 17 if the encrypted mes- 
sage is 3185 2038 2460 2550? (To decrypt, first find the 
decryption exponent d, which is the inverse of e = 17 
modulo 52 - 60.) 

27. What is the original message encrypted using the RSA 
system with n = 43 -59 and e = 13 if the encrypted mes- 
sage is 0667 1947 0671? (To decrypt, first find the de- 
cryption exponent d which is the inverse of e = 13 mod- 
ulo 42 - 58.) 


*28. Suppose that (n,e) is an RSA encryption key, 
with n = pq, where p and q are large primes and 
gcd(e, (p — 1X(q — 1)) = 1. Furthermore, suppose that 
d is an inverse of e modulo (p — 1)(g — 1). Suppose that 
C = M° (mod pq). In the text we showed that RSA de- 
cryption, that is, the congruence C! = M (mod pq) holds 
when gcd(M, pq) = 1. Show that this decryption congru- 
ence also holds when gcd(M, pq) > 1. [Hint: Use con- 
gruences modulo p and modulo q and apply the Chinese 
remainder theorem. ] 

29. Describe the steps that Alice and Bob follow when they 
use the Diffie-Hellman key exchange protocol to generate 
a shared key. Assume that they use the prime p = 23 and 
take a = 5, which is a primitive root of 23, and that Alice 
selects k; = 8 and Bob selects k, = 5. (You may want to 
use some computational aid.) 

30. Describe the steps that Alice and Bob follow when they 
use the Diffie-Hellman key exchange protocol to gener- 
ate a shared key. Assume that they use the prime p = 101 
and take a = 2, which is a primitive root of 101, and that 
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Alice selects k; = 7 and Bob selects k, = 9. (You may 
want to use some computational aid.) 


In Exercises 31-32 suppose that Alice and Bob have these 
public keys and corresponding private keys: (Malice, CAlice) = 
(2867, 7) = (61 - 47,7),  dajice = 1183 and (ngob €Bob) = 
(3127, 21) = (59 - 53, 21), dg, = 1149. First express your 
answers without carrying out the calculations. Then, using a 
computational aid, if available, perform the calculation to get 
the numerical answers. 


31. Alice wants to send to all her friends, including Bob, the 
message “SELL EVERYTHING?” so that he knows that 
she sent it. What should she send to her friends, assuming 
she signs the message using the RSA cryptosystem. 


32. Alice wants to send to Bob the message “BUY NOW” so 
that he knows that she sent it and so that only Bob can 
read it. What should she send to Bob, assuming she signs 
the message and then encrypts it using Bob’s public key? 


33. We describe a basic key exchange protocol using private 
key cryptography upon which more sophisticated proto- 
cols for key exchange are based. Encryption within the 
protocol is done using a private key cryptosystem (such 
as AES) that is considered secure. The protocol involves 
three parties, Alice and Bob, who wish to exchange a key, 
and a trusted third party Cathy. Assume that Alice has a 
secret key kaj. that only she and Cathy know, and Bob 
has a secret key kgop which only he and Cathy know. The 
protocol has three steps: 


(i) Alice sends the trusted third party Cathy the message 
“request a shared key with Bob” encrypted using Alice’s 
key ky 


lice* 


(ii) Cathy sends back to Alice a key Kajice gop» Which she 
generates, encrypted using the key k,),.., followed by this 
same key Kajice Bob» encrypted using Bob’s key, kgob- 


(iii) Alice sends to Bob the key kitice Bob encrypted using 
kop, Known only to Bob and to Cathy. 


Explain why this protocol allows Alice and Bob to share 

the secret key Kj:c¢ Bob» Known only to them and to Cathy. 
The Paillier cryptosystem is a public key cryptosystem de- 
scribed in 1999 by P. Paillier, used in some electronic voting 
systems. A public key (n, g) and a corresponding private key 
(p, q) are created by randomly selecting primes p and q so that 
gcd(pq, A) = 1, where A = (p — 1)(q — 1), and a nonzero ele- 
ment g of Z, with ged((g#™4” — 1)/n, n) = 1. To encrypt 
a message m € Z, a random nonzero element r of Z, is first 
selected, and then used to find c = gr" mod n?. 


34. a) Show that we can take p = 149, q = 179, and g = 5 
to generate a public key for the Paillier cryptosystem 
by checking that all the conditions required for these 
parameters hold, and find the public and private keys 
that are generated. 

b) Find the ciphertext corresponding to the plaintext 
m = 67 where we choose r = 81. 


35. Show that the Paillier cryptosystem is additively homo- 
morphic. 
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Key Terms and Results 


TERMS 


a | b (a divides b): there is an integer c such that b = ac 

a and b are congruent modulo m: m divides a — b 

modular arithmetic: arithmetic done modulo an integer 
m>2 

prime: an integer greater than 1 with exactly two positive 
integer divisors 

composite: an integer greater than | that is not prime 

Mersenne prime: a prime of the form 2? — 1, where p is prime 


gcd(a, b) (greatest common divisor of a and b): the largest 
integer that divides both a and b 

relatively prime integers: integers a and b such that 
gcd(a, b) = 1 

pairwise relatively prime integers: a set of integers with the 
property that every pair of these integers is relatively prime 


Icm(a, b) (least common multiple of a and b): the smallest 
positive integer that is divisible by both a and b 

a mod b: the remainder when the integer a is divided by the 
positive integer b 

a = b (mod m) (a is congruent to b modulo m): a — b is 
divisible by m 

n = (a,.0,_1 «+. @19),: the base b representation of n 

binary representation: the base 2 representation of an integer 

octal representation: the base 8 representation of an integer 

hexadecimal representation: the base 16 representation of 
an integer 

linear combination of a and b with integer coefficients: an 
expression of the form sa + tb, where s and t are integers 

Bézout coefficients of a and b: integers s and ¢ such that the 
Bézout identity sa + th = gcd(a, b) holds 

inverse of a modulo m: an integer a such that aa = 1 (mod m) 

linear congruence: a congruence of the form ax = b (mod m), 
where x is an integer variable 

pseudoprime to the base b: a composite integer n such that 
b"! = 1 (mod n) 

Carmichael number: a composite integer n such that n is a 
pseudoprime to the base b for all positive integers b with 
gcd(b, n) = 1 

primitive root of a prime p: an integer r in Z, such that every 
integer not divisible by p is congruent modulo p to a power 
of r 

discrete logarithm of a to the base r modulo p: the integer e 
with 0 < e < p — 1 such that r° = a (mod p) 

encryption: the process of making a message secret 

decryption: the process of returning a secret message to its 
original form 

encryption key: a value that determines which of a family of 
encryption functions is to be used 

shift cipher: a cipher that encrypts the plaintext letter p as 
(p + k) mod m for an integer k 


affine cipher: a cipher that encrypts the plaintext letter p as 
(ap + b) mod m for integers a and b with gcd(a, 26) = 1 


character cipher: a cipher that encrypts characters one by one 


block cipher: a cipher that encrypts blocks of characters of a 
fixed size 


cryptanalysis: the process of recovering the plaintext from 
ciphertext without knowledge of the encryption method, or 
with knowledge of the encryption method, but not the key 


cryptosystem: a five-tuple (P, C, K, €, D) where P is the set 
of plaintext messages, C is the set of ciphertext messages, 
K is the set of keys, € is the set of encryption functions, 
and D is the set of decryption functions 


private key encryption: encryption where both encryption 
keys and decryption keys must be kept secret 


public key encryption: encryption where encryption keys are 
public knowledge, but decryption keys are kept secret 


RSA cryptosystem: the cryptosystem where P and C are 
both Z,,, K is the set of pairs k = (n, e) where n = pq 
where p and q are large primes and e is a positive integer, 
E,(p) = p° modn, and D,(c) = c4modn where d is the 
inverse of e modulo (p — 1)(g — 1) 


key exchange protocol: a protocol used for two parties to 
generate a shared key 


digital signature: a method that a recipient can use to deter- 
mine that the purported sender of a message actually sent 
the message 


fully homomorphic cryptosystem: a cryptosystem that al- 
lows arbitrary computations to be run on encrypted data so 
that the ouptut is the encryption of the unencrypted output 
of the unencrypted input 


RESULTS 


division algorithm: Let a and d be integers with d positive. 
Then there are unique integers q and r with O < r < d such 
that a = dq +r. 


Let b be an integer greater than 1. Then if n is a posi- 
tive integer, it can be expressed uniquely in the form 
n= a,b + a;b! + -+ ab + ap. 

The algorithm for finding the base b expansion of an integer 
(see Algorithm 1 in Section 4.2) 


The conventional algorithms for addition and multiplication 
of integers (given in Section 4.2) 

The fast modular exponentiation algorithm (see Algorithm 5 
in Section 4.2) 

Euclidean algorithm: for finding greatest common divisors 
by successively using the division algorithm (see Algo- 
rithm 1 in Section 4.3) 

Bézout’s theorem: If a and b are positive integers, then 
gcd(a, b) is a linear combination of a and b. 

sieve of Eratosthenes: a procedure for finding all primes not 
exceeding a specified number n, described in Section 4.3 


fundamental theorem of arithmetic: Every positive integer 


can be written uniquely as the product of primes, where the 
prime factors are written in order of increasing size. 


If a and b are positive integers, then ab = gcd(a, b); Icm(a, b). 
If mis a positive integer and gcd(a, m) = 1, then a has a unique 


inverse modulo m. 


Review Questions 


= 


a 


10. 


Find 210 div 17 and 210 mod 17. 

a) Define what it means for a and b to be congruent mod- 
ulo 7. 

b) Which pairs of the integers —11, —8, —7, —1, 0, 3, and 
17 are congruent modulo 7? 

c) Show that if a and b are congruent modulo 7, then 
10a + 13 and —4b + 20 are also congruent modulo 7. 

Show that if a = b (mod m) and c = d (mod m), then 

a+c =b +d (mod m). 

Describe a procedure for converting decimal (base 10) 

expansions of integers into hexadecimal expansions. 

Convert (1101 1001 0101 1011), to octal and hexadeci- 

mal representations. 

Convert (7206); and (AOEB),, to a binary representation. 

State the fundamental theorem of arithmetic. 

a) Describe a procedure for finding the prime factoriza- 
tion of an integer. 

b) Use this procedure to find the prime factorization of 
80,707. 

a) Define the greatest common divisor of two integers. 

b) Describe at least three different ways to find the great- 
est common divisor of two integers. When does each 
method work best? 

c) Find the greatest common divisor of 1,234,567 and 
7,654,321. 

d) Find the greatest common divisor of 2335577°11 and 
Boer is: 

a) How can you find a linear combination (with integer 
coefficients) of two integers that equals their greatest 
common divisor? 


Supplementary Exercises 


1. 


The odometer on a car goes to up 100,000 miles. The 
present owner of a car bought it when the odometer read 
43,179 miles. He now wants to sell it; when you examine 
the car for possible purchase, you notice that the odome- 
ter reads 89,697 miles. What can you conclude about how 
many miles he drove the car, assuming that the odometer 
always worked correctly? 


. a) Explain why n div 7 equals the number of complete 


weeks in n days. 


b) Explain why n div 24 equals the number of complete 
days in n hours. 
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Chinese remainder theorem: A system of linear congruences 
modulo pairwise relatively prime integers has a unique so- 
lution modulo the product of these moduli. 


Fermat’s little theorem: If p is prime and p ja, then 
a?! = 1 (mod p). 


11. 


12. 


13. 


14. 
15. 
16. 
17. 


18. 


19. 


20. 


#5. 


b) Express gcd(84, 119) as a linear combination of 84 
and 119. 


a) What does it mean for a to be an inverse of 
a modulo m? 

b) How can you find an inverse of a modulo m when m 
is a positive integer and gcd(a, m) = 1? 

c) Find an inverse of 7 modulo 19. 

a) How can an inverse of a modulo m be used to solve 
the congruence ax = b (mod m) when gcd(a, m) = 1? 

b) Solve the linear congruence 7x = 13 (mod 19). 

a) State the Chinese remainder theorem. 

b) Find the solutions to the system x = 1 (mod 4), x = 
2 (mod 5), and x = 3 (mod 7). 

Suppose that 2”-! = 1 (mod n). Is n necessarily prime? 

Use Fermat’s little theorem to evaluate 9°% mod 19. 

Explain how the check digit is found for a 10-digit ISBN. 

Encrypt the message APPLES AND ORANGES using a 

shift cipher with key k = 13. 

a) What is the difference between a public key and a pri- 
vate key cryptosystem? 

b) Explain why using shift ciphers is a private key sys- 
tem. 

c) Explain why the RSA cryptosystem is a public key 
system. 

Explain how encryption and decryption are done in the 

RSA cryptosystem. 


Describe how two parties can share a secret key using the 
Diffie-Hellman key exchange protocol. 


. Find four numbers congruent to 5 modulo 17. 


. Show that if a and d are positive integers, then there are 


integers q and r such that a = dq + r, where —d /2 < r < 
d/2. 
Show that if ac = bc (mod m), where a,b,c, and 


m are integers with m > 2, and d = gcd(m, c), then 
a = b (mod m/d). 


. Show that the sum of the squares of two odd integers can- 


not be the square of an integer. 


. Show that if n? + 1 is a perfect square, where n is an in- 


teger, then n is even. 
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8. 


10. 


12. 


13. 


** 14, 


15. 


16. 


17. 


18. 


*19. 


20. 
21. 


22. 


* 23. 


* 24. 


25. 


26. 


27. 


Prove that there are no solutions in integers x and y to 
the equation x? — 5y? = 2. [Hint: Consider this equation 
modulo 5.] 


. Develop a test for divisibility of a positive integer n by 8 


based on the binary expansion of n. 


Develop a test for divisibility of a positive integer n by 3 
based on the binary expansion of n. 


. Devise an algorithm for guessing a number between | and 


2” — 1 by successively guessing each bit in its binary ex- 
pansion. 
Determine the complexity, in terms of the number of 
guesses, needed to determine a number between 1 and 
2” — 1 by successively guessing the bits in its binary ex- 
pansion. 


Show that an integer is divisible by 9 if and only if the 
sum of its decimal digits is divisible by 9. 

Show that if a and b are positive irrational numbers such 
that 1/a + 1/b = 1, then every positive integer can be 
uniquely expressed as either [ka] or |kb] for some posi- 
tive integer k. 

Prove there are infinitely many primes by showing that 
Q,,=n!+1 must have a prime factor greater than n 
whenever n is a positive integer. 

Find a positive integer n for which Q, =n! +1 is not 
prime. 

Use Dirichlet’s theorem, which states there are infinitely 
many primes in every arithmetic progression ak + b 
where gcd(a, b) = 1, to show that there are infinitely 
many primes that have a decimal expansion ending 
with a 1. 

Prove that if n is a positive integer such that the sum of 
the divisors of n is n + 1, then n is prime. 

Show that every integer greater than 11 is the sum of two 
composite integers. 


Find the five smallest consecutive composite integers. 


Show that Goldbach’s conjecture, which states that ev- 
ery even integer greater than 2 is the sum of two primes, 
is equivalent to the statement that every integer greater 
than 5 is the sum of three primes. 


Find an arithmetic progression of length six beginning 
with 7 that contains only primes. 

Prove that if f(x) is a nonconstant polynomial with inte- 
ger coefficients, then there is an integer y such that f(y) is 
composite. [Hint: Assume that f(x) = p is prime. Show 
that p divides f (xọ + kp) for all integers k. Obtain a con- 
tradiction of the fact that a polynomial of degree n, where 
n > 1, takes on each value at most n times.] 

How many zeros are at the end of the binary expansion 
of (100,9)!? 

Use the Euclidean algorithm to find the greatest common 
divisor of 10,223 and 33,341. 


How many divisions are required to find gcd(144, 233) 
using the Euclidean algorithm? 

Find ged(2n + 1, 3n + 2), where n is a positive integer. 
[Hint: Use the Euclidean algorithm. ] 


28. 


29. 


30. 


31. 


32. 


a) Show that if a and b are positive inte- 
gers with a>b, then gcd(a,b)=a if a=b, 
gcd(a,b) = 2 gcd(a/2,b/2) if a and b are even, 
gcd(a, b) = gcd(a/2, b) if a is even and b is odd, and 
gcd(a, b) = gcd(a — b, b) if both a and b are odd. 

b) Explain how to use part (a) to construct an algorithm 
for computing the greatest common divisor of two 
positive integers that uses only comparisons, subtrac- 
tions, and shifts of binary expansions, without using 
any divisions. 

c) Find gcd(1202, 4848) using this algorithm. 


Adapt the proof that there are infinitely many primes 
(Theorem 3 in Section 4.3) to show that there are in- 
finitely many primes in the arithmetic progression 6k + 5, 
ke, Dy nade 


Explain why you cannot directly adapt the proof that there 
are infinitely many primes (Theorem 3 in Section 4.3) to 
show that there are infinitely many primes in the arith- 
metic progression 3k + 1, k = 1,2,.... 

Explain why you cannot directly adapt the proof that there 
are infinitely many primes (Theorem 3 in Section 4.3) to 
show that there are infinitely many primes in the arith- 
metic progression 4k + 1, k = 1,2,.... 

Show that if the smallest prime factor p of the positive 


integer n is larger than vn, then n/p is prime or equal 
to 1. 


A set of integers is called mutually relatively prime if the 
greatest common divisor of these integers is 1. 


33. 


34. 


*35, 
36. 


37. 


*38. 


39. 


40. 


41. 


Determine whether the integers in each of these sets are 
mutually relatively prime. 

a) 8,10, 12 b) 12, 15, 25 

c) 15, 21, 28 d) 21, 24, 28, 32 

Find a set of four mutually relatively prime integers such 
that no two of them are relatively prime. 

For which positive integers n is n* + 4” prime? 

Show that the system of congruences x = 2 (mod 6) and 
x = 3 (mod 9) has no solutions. 


Find all solutions of the system of congruences x = 
4 (mod 6) and x = 13 (mod 15). 


a) Show that the system of congruences x = a; (mod m) 
and x = a, (mod m,), where a, dy, m;, and m, are in- 
tegers with m, > 0 and m, > 0, has a solution if and 
only if gcd(m,, m) | ay — az. 

b) Show that if the system in part (a) has a solution, then 
it is unique modulo Ilem(m, m). 


Prove that 30 divides n? —n for every nonnegative 
integer n. 

Prove that n!? — 1 is divisible by 35 for every integer n 
for which gcd(n, 35) = 1. 

Show that if p and q are distinct prime numbers, then 
pt! + q7! = 1 (mod pq). 


The check digit aj% for an ISBN-13 with initial digits 
dy ... ay is determined by the congruence (a, + 43 + + + 
443) + 3(a, + a4 + +++ + a42) = 0 (mod 10). 


42. Determine whether each of these 13-digit numbers is a 

valid ISBN-13. 
a) 978-0-073-20679-1 
b) 978-0-45424-521-1 
c) 978-3-16-148410-0 
d) 978-0-201-10179-9 

43. Show that the check digit of an ISBN-13 can always de- 
tect a single error. 

44. Show that there are transpositions of two digits that are 
not detected by an ISBN-13. 

A routing transit number (RTN) is a bank code used 

in the United States which appears on the bottom of 

checks. The most common form of an RTN has nine dig- 
its, where the last digit is a check digit. If d,d,...dy is 

a valid RTN, the congruence 3(d, + d4 + d7) + 7(d, + d; + 

dg) + (d; + dg + dy) = 0 (mod 10) must hold. 

45. Show that if d;d, ... dọ is a valid RTN, then dy = 7(d, + 
dy + dz) + 3(d, + d; + dg) + 9(d; + dẹ)mod10. Fur- 
thermore, use this formula to find the check digit that 
follows the eight digits 11100002 in a valid RTN. 

46. Show that the check digit of an RTN can detect all single 
errors and determine which transposition errors an RTN 
check digit can catch and which ones it cannot catch. 

47. The encrypted version of a message is LIMKG MG- 
MXF QEXMW If it was encrypted using the affine cipher 
f(p) = (7p + 10) mod 26, what was the original message? 


Computer Projects 


Write programs with these inputs and outputs. 


1. Given integers n and b, each greater than 1, find the base 
b expansion of this integer. 


2. Given the positive integers a, b, and m with m > 1, find 
a’ mod m. 


3. Given a positive integer, find the Cantor expansion of this 
integer (see the preamble to Exercise 54 of Section 4.2). 


4. Given a positive integer, determine whether it is prime 
using trial division. 

5. Given a positive integer, find the prime factorization of 
this integer. 


6. Given two positive integers, find their greatest common 
divisor using the Euclidean algorithm. 


7. Given two positive integers, find their least common mul- 
tiple. 

8. Given positive integers a and b, find Bézout coefficients 
s and t of a and b. 


9. Given relatively prime positive integers a and b, find an 
inverse of a modulo b. 


10. Given n linear congruences modulo pairwise relatively 
prime moduli, find the simultaneous solution of these 
congruences modulo the product of these moduli. 
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Autokey ciphers are ciphers where the nth letter of the plain- 
text is shifted by the numerical equivalent of the nth letter of a 
keystream. The keystream begins with a seed letter; its subse- 
quent letters are constructed using either the plaintext or the 
ciphertext. When the plaintext is used, each character of the 
keystream, after the first, is the previous letter of the plaintext. 
When the ciphertext is used, each subsequent character of the 
keystream, after the first, is the previous letter of the ciphertext 
computed so far. In both cases, plaintext letters are encrypted 
by shifting each character by the numerical equivalent of the 
corresponding keystream letter. 


48. Use the autokey cipher to encrypt the message NOW IS 
THE TIME TO DECIDE (ignoring spaces) using 
a) the keystream with seed X followed by letters of the 
plaintext. 


b) the keystream with seed X followed by letters of the 
ciphertext. 


49. Use the autokey cipher to encrypt the message THE 
DREAM OF REASON (ignoring spaces) using 
a) the keystream with seed X followed by letters of the 
plaintext. 


b) the keystream with seed X followed by letters of the 
ciphertext. 


11. Given a positive integer N, a modulus m, a multiplier a, 
an increment c, and a seed x), where O < a < m, 0 < c < 
m, and 0 < x) < m, generate the sequence of N pseudo- 
random numbers using the linear congruential generator 
X +1 = (ax, +c) mod m. 


12. Given a set of identification numbers, use a hash func- 
tion to assign them to memory locations where there are 
k memory locations. 


13. Compute the check digit when given the first nine digits 
of an ISBN-10. 


14. Given a message and a positive integer k less than 26, en- 
crypt this message using the shift cipher with key k; and 
given a message encrypted using a shift cipher with key 
k, decrypt this message. 


15. Given a message and positive integers a and b less than 
26 with gcd(a, 26), encrypt this message using an affine 
cipher with key (a, b); and given a message encrypted us- 
ing the affine cipher with key (a, b), decrypt this message, 
by first finding the decryption key and then applying the 
appropriate decryption transformation. 


16. Find the original plaintext message from the ciphertext 
message produced by encrypting the plaintext message 
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#17. 


18. 


using a shift cipher. Do this using a frequency count of 
letters in the ciphertext. 


Construct a valid RSA encryption key by finding two 
primes p and q with 200 digits each and an integer e > 1 
relatively prime to (p — 1)(q — 1). 


Given a message and an integer n = pq where p and q 
are odd primes and an integer e > 1 relatively prime to 
(p — 1)(q — 1), encrypt the message using the RSA cryp- 
tosystem with key (n, e). 


Computations and Explorations 


19. 


20. 


21. 


22. 


Given a valid RSA key (n, e), and the primes p and q with 
n = pq, find the associated decryption key d. 

Given a message encrypted using the RSA cryptosystem 
with key (n, e) and the associated decryption key d, de- 
crypt this message. 

Generate a shared key using the Diffie-Hellman key ex- 
change protocol. 

Given the RSA public and private keys of two parties, 
send a signed secret message from one of the parties to 
the other. 


Use a computational program or programs you have written to do these exercises. 


. Determine whether 2? — 1 is prime for each of the primes 


not exceeding 100. 


. Test a range of large Mersenne numbers 2? — 1 to deter- 


mine whether they are prime. (You may want to use soft- 
ware from the GIMPS project.) 


. Determine whether Q, = piP Pa +1 is prime where 


Pi Pa +++» Pa are the n smallest primes, for as many pos- 
itive integers n as possible. 


. Look for polynomials in one variables whose values at 


long runs of consecutive integers are all primes. 


. Find as many primes of the form n? + 1 where n is a pos- 


itive integer as you can. It is not known whether there are 
infinitely many such primes. 


Writing Projects 


Respond to these with essays using outside sources. 


. Describe the Lucas—Lehmer test for determining whether 


a Mersenne number is prime. Discuss the progress of the 
GIMPS project in finding Mersenne primes using this 
test. 


. Explain how probabilistic primality tests are used in prac- 


tice to produce extremely large numbers that are almost 
certainly prime. Do such tests have any potential draw- 
backs? 


. The question of whether there are infinitely many 


Carmichael numbers was solved recently after being open 
for more than 75 years. Describe the ingredients that went 
into the proof that there are infinitely many such numbers. 


. Summarize the current status of factoring algorithms in 


terms of their complexity and the size of numbers that 
can currently be factored. When do you think that it will 
be feasible to factor 200-digit numbers? 


. Describe the algorithms that are actually used by modern 


computers to add, subtract, multiply, and divide positive 
integers. 


6. 
7. 


10. 


Find 10 different primes each with 100 digits. 


How many primes are there less than 1,000,000, less than 
10,000,000, and less than 100,000,000? Can you propose 
an estimate for the number of primes less than x where x 
is a positive integer? 


. Find a prime factor of each of 10 different 20-digit odd in- 


tegers, selected at random. Keep track of how long it takes 
to find a factor of each of these integers. Do the same thing 
for 10 different 30-digit odd integers, 10 different 40-digit 
odd integers, and so on, continuing as long as possible. 


. Find all pseudoprimes to the base 2 that do not exceed 


10,000. 


. Describe the history of the Chinese remainder theorem. 


Describe some of the relevant problems posed in Chi- 
nese and Hindu writings and how the Chinese remainder 
theorem applies to them. 


. When are the numbers of a sequence truly random num- 


bers, and not pseudorandom? What shortcomings have 
been observed in simulations and experiments in which 
pseudorandom numbers have been used? What are the 
properties that pseudorandom numbers can have that ran- 
dom numbers should not have? 


. Explain how a check digit is found for an International 


Bank Account Number (IBAN) and discuss the types of 
errors that can be found using this check digit. 


. Describe the Luhn algorithm for finding the check digit 


of a credit card number and discuss the types of errors 
that can be found using this check digit. 


Show how a congruence can be used to tell the day of the 
week for any given date. 


11. 


12. 


13. 


Describe how public key cryptography is being applied. 
Are the ways it is applied secure given the status of fac- 
toring algorithms? Will information kept secure using 
public key cryptography become insecure in the future? 


Describe how public key cryptography can be used to 
produce signed secret messages so that the recipient is 
relatively sure the message was sent by the person ex- 
pected to have sent it. 


Describe the Rabin public key cryptosystem, explaining 
how to encrypt and how to decrypt messages and why it 
is suitable for use as a public key cryptosystem. 


*14, 


15. 


*16. 
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Explain why it would be unsuitable to use p, where p 
is a large prime, as the modulus for encryption in the 
RSA cryptosystem. That is, explain how someone could, 
without excessive computation, find a private key from 
the corresponding public key if the modulus were a large 
prime, rather than the product of two large primes. 


Explain what is meant by a cryptographic hash function. 
What are the important properties such a function must 
have? 


Explain the steps that Gentry used to construct a fully 
homomorphic cryptosystem. 
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Induction and Recursion 


M any mathematical statements assert that a property is true for all positive integers. Ex- 
amples of such statements are that for every positive integer n: n! < n”, nò — nis divis- 
ible by 3; a set with n elements has 2” subsets; and the sum of the first n positive integers is 
n(n + 1)/2. A major goal of this chapter, and the book, is to provide a thorough understanding 
of mathematical induction, which is used to prove results of this kind. 

Proofs using mathematical induction have two parts. First, they show that the statement 
holds for the positive integer 1. Second, they show that if the statement holds for a positive 
integer then it must also hold for the next larger integer. Mathematical induction is based on the 
rule of inference that tells us that if P(1) and Vk(P(k) > P(k + 1)) are true for the domain of 
positive integers, then YnP(n) is true. Mathematical induction can be used to prove a tremendous 
variety of results. Understanding how to read and construct proofs by mathematical induction 
is a key goal of learning discrete mathematics. 

In Chapter 2 we explicitly defined sets and functions. That is, we described sets by listing 
their elements or by giving some property that characterizes these elements. We gave formu- 
lae for the values of functions. There is another important way to define such objects, based on 
mathematical induction. To define functions, some initial terms are specified, and a rule is given 
for finding subsequent values from values already known. (We briefly touched on this sort of 
definition in Chapter 2 when we showed how sequences can be defined using recurrence rela- 
tions.) Sets can be defined by listing some of their elements and giving rules for constructing 
elements from those already known to be in the set. Such definitions, called recursive defini- 
tions, are used throughout discrete mathematics and computer science. Once we have defined 
a set recursively, we can use a proof method called structural induction to prove results about 
this set. 

When a procedure is specified for solving a problem, this procedure must always solve 
the problem correctly. Just testing to see that the correct result is obtained for a set of input 
values does not show that the procedure always works correctly. The correctness of a procedure 
can be guaranteed only by proving that it always yields the correct result. The final section of 
this chapter contains an introduction to the techniques of program verification. This is a formal 
technique to verify that procedures are correct. Program verification serves as the basis for 
attempts under way to prove in a mechanical fashion that programs are correct. 


Mathematical Induction 


5.1.1 Introduction 


Suppose that we have an infinite ladder, as shown in Figure 1, and we want to know whether we 
can reach every step on this ladder. We know two things: 


1. We can reach the first rung of the ladder. 
2. If we can reach a particular rung of the ladder, then we can reach the next rung. 


Can we conclude that we can reach every rung? By (1), we know that we can reach the first 
rung of the ladder. Moreover, because we can reach the first rung, by (2), we can also reach the 
second rung; it is the next rung after the first rung. Applying (2) again, because we can reach the 
second rung, we can also reach the third rung. Continuing in this way, we can show that we can 
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Assessment » 


We can reach step k + 1 if 
we can reach step k 


We can reach 
step 1 


FIGURE 1 Climbing an infinite ladder. 


reach the fourth rung, the fifth rung, and so on. For example, after 100 uses of (2), we know that 
we can reach the 101st rung. But can we conclude that we are able to reach every rung of this 
infinite ladder? The answer is yes, something we can verify using an important proof technique 
called mathematical induction. That is, we can show that the statement that we can reach the 
nth rung of the ladder is true for all positive integers n. 

Mathematical induction is an extremely important proof technique that can be used to prove 
assertions of this type. As we will see in this section and in subsequent sections of this chapter 
and later chapters, mathematical induction is used extensively to prove results about a large 
variety of discrete objects. For example, it is used to prove results about the complexity of 
algorithms, the correctness of certain types of computer programs, theorems about graphs and 
trees, as well as a wide range of identities and inequalities. 

In this section, we will describe how mathematical induction can be used and why it is a 
valid proof technique. It is extremely important to note that mathematical induction can be used 
only to prove results obtained in some other way. It is not a tool for discovering formulae or 
theorems. 


5.1.2 Mathematical Induction 


In general, mathematical induction* can be used to prove statements that assert that P(n) is 
true for all positive integers n, where P(n) is a propositional function. A proof by mathematical 


*Unfortunately, using the terminology “mathematical induction” clashes with the terminology used to describe different types 
of reasoning. In logic, deductive reasoning uses rules of inference to draw conclusions from premises, whereas inductive 
reasoning makes conclusions only supported, but not ensured, by evidence. Mathematical proofs, including arguments that 
use mathematical induction, are deductive, not inductive. 
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induction has two parts, a basis step, where we show that P(1) is true, and an inductive step, 
where we show that for all positive integers k, if P(k) is true, then P(k + 1) is true. 


PRINCIPLE OF MATHEMATICAL INDUCTION To prove that P(n) is true for all positive 
integers n, where P(n) is a propositional function, we complete two steps: 


BASIS STEP: We verify that P(1) is true. 


INDUCTIVE STEP: We show that the conditional statement P(k) > P(k + 1) is true for all 
positive integers k. 


To complete the inductive step of a proof using the principle of mathematical induction, we 
assume that P(k) is true for an arbitrary positive integer k and show that under this assumption, 
P(k + 1) must also be true. The assumption that P(k) is true is called the inductive hypothesis. 
Once we complete both steps in a proof by mathematical induction, we have shown that P(n) is 
true for all positive integers n, that is, we have shown that YnP(n) is true where the quantification 
is over the set of positive integers. In the inductive step, we show that Vk(P(k) > P(k + 1)) is 
true, where again, the domain is the set of positive integers. 
Expressed as a rule of inference, this proof technique can be stated as 


(PU) AVK(P(A) > P(k + 1))) > YnP(n), 


when the domain is the set of positive integers. Because mathematical induction is such an 
important technique, it is worthwhile to explain in detail the steps of a proof using this technique. 
The first thing we do to prove that P(n) is true for all positive integers n is to show that P(1) is 
true. This amounts to showing that the particular statement obtained when n is replaced by 1 
in P(n) is true. Then we must show that P(k) > P(k + 1) is true for every positive integer k. To 
prove that this conditional statement is true for every positive integer k, we need to show that 
P(k + 1) cannot be false when P(x) is true. This can be accomplished by assuming that P(x) is 
true and showing that under this hypothesis P(k + 1) must also be true. 


Remark: In a proof by mathematical induction it is not assumed that P(x) is true for all positive 
integers! It is only shown that if it is assumed that P(k) is true, then P(k + 1) is also true. Thus, 
a proof by mathematical induction is not a case of begging the question, or circular reasoning. 


After completing the basis and inductive steps of a proof that P(n) is true for all positive integers 
n, we know that P(1) is true. That is what is shown in the basis step. We can conclude that P(2) is 
true, because we know that P(1) is true and from the inductive step we know that P(1) > P(2). 
Furthermore, we know that P(3) is true because P(2) is true and we know that P(2) > P(3) 
from the inductive step. Continuing along these lines using a finite number of implications, we 
can show that P(n) is true for any particular positive integer n. 


HISTORICAL NOTE The first known use of mathematical induction is in the work of the sixteenth-century 
mathematician Francesco Maurolico (1494 — 1575). Maurolico wrote extensively on the works of classical math- 
ematics and made many contributions to geometry and optics. In his book Arithmeticorum Libri Duo, Maurolico 
presented a variety of properties of the integers together with proofs of these properties. To prove some of these 
properties, he devised the method of mathematical induction. His first use of mathematical induction in this 
book was to prove that the sum of the first n odd positive integers equals n?. Augustus De Morgan is credited 
with the first presentation in 1838 of formal proofs using mathematical induction, as well as introducing the ter- 
minology “mathematical induction.” Maurolico’s proofs were informal and he never used the word “induction.” 
See [Gu10] to learn more about the history of the method of mathematical induction. 
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FIGURE 2 Illustrating how mathematical induction works using dominoes. 


WAYS TO REMEMBER HOW MATHEMATICAL INDUCTION WORKS Thinking of the 
infinite ladder and the rules for reaching steps can help you remember how mathematical in- 
duction works. Note that statements (1) and (2) for the infinite ladder are exactly the basis step 
and inductive step, respectively, of the proof that P(n) is true for all positive integers n, where 
P(n) is the statement that we can reach the nth rung of the ladder. Consequently, we can invoke 
mathematical induction to conclude that we can reach every rung. 

Another way to illustrate the principle of mathematical induction is to consider an infinite 
row of dominoes, labeled 1, 2, 3, ..., n, ... , where each domino is standing up. Let P(n) be the 
proposition that domino n is knocked over. If the first domino is knocked over—that is, if P(1) 
is true—and if, whenever the kth domino is knocked over, it also knocks the (k + 1)st domino 
over—that is, if P(k) > P(k + 1) is true for all positive integers k—then all the dominoes are 
knocked over. This is illustrated in Figure 2. 


5.1.3 Why Mathematical Induction is Valid 


Why is mathematical induction a valid proof technique? The reason comes from the well- 
ordering property, listed in Appendix 1 as an axiom for the set of positive integers, which states 
that every nonempty subset of the set of positive integers has a least element. So, suppose we 
know that P(1) is true and that the proposition P(k) — P(k + 1) is true for all positive integers k. 
To show that P(n) must be true for all positive integers n, assume that there is at least one posi- 
tive integer n for which P(n) is false. Then the set S of positive integers n for which P(n) is false 
is nonempty. Thus, by the well-ordering property, S has a least element, which will be denoted 
by m. We know that m cannot be 1, because P(1) is true. Because m is positive and greater 
than 1, m — 1 is a positive integer. Furthermore, because m — 1 is less than m, it is not in S, so 
P(m — 1) must be true. Because the conditional statement P(m — 1) > P(m) is also true, it must 
be the case that P(m) is true. This contradicts the choice of m. Hence, P(n) must be true for 
every positive integer n. 


Remark: In this book we take the well-ordering property for the positive integers as an axiom. 
We proved that mathematical induction is a valid proof technique. Instead, we could have taken 
the principle of mathematical induction as an axiom and proved that the positive integers are well 
ordered. That is, the well-ordering property for positive integers and the principle of mathematical 
induction are equivalent. (In Section 5.2 we will present examples of proofs that use the well-ordering 
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property directly. Also, Exercise 41 in that section asks for a proof that the well-ordering property 
for positive integers is a consequence of the principle of mathematical induction.) 


5.1.4 Choosing the Correct Basis Step 


Mathematical induction can be used to prove theorems other than those of the form “P(7) 
is true for all positive integers n.” Often, we will need to show that P(n) is true for n = 


b, b+1, b+2,..., where b is an integer other than 1. We can use mathematical induction 
to accomplish this, as long as we change the basis step by replacing P(1) with P(d). In other 
words, to use mathematical induction to show that P(n) is true for n = b, b + 1,b+2,..., where 


b is an integer other than 1, we show that P(b) is true in the basis step. In the inductive step, 
we show that the conditional statement P(k) > P(k + 1) is true fork = b, b + 1, b + 2, ... . Note 
that b can be negative, zero, or positive. Following the domino analogy we used earlier, imagine 
that we begin by knocking down the bth domino (the basis step), and as each domino falls, it 
knocks down the next domino (the inductive step). We leave it to the reader to show that this 
form of induction is valid (see Exercise 85). 

We will illustrate this notion in Example 3, which states that a summation formula is valid 
for all nonnegative integers. In this example, we need to prove that P(n) is true forn = 0, 1, 2, .... 
So, the basis step in Example 3 will show that P(O) is true. 


5.1.5 Guidelines for Proofs by Mathematical Induction 


Examples 1-14 will illustrate how to use mathematical induction to prove a diverse collection 
of theorems. Each of these examples includes all the elements needed in a proof by mathemat- 
ical induction. We will also present an example of an invalid proof by mathematical induction. 
Before we give these proofs, we will provide some useful guidelines for constructing correct 
proofs by mathematical induction. 


Template for Proofs by Mathematical Induction 


1. Express the statement that is to be proved in the form “for all n > b, P(n)” for a fixed 
integer b. For statements of the form “P(n) for all positive integers n,” let b = 1, and for 
statements of the form “P(n) for all nonnegative integers n,” letb = 0. For some statements 
of the form P(n), such as inequalities, you may need to determine the appropriate value 
of b by checking the truth values of P(n) for small values of n, as is done in Example 6. 

2. Write out the words “Basis Step.” Then show that P(b) is true, taking care that the correct 
value of b is used. This completes the first part of the proof. 

3. Write out the words “Inductive Step” and state, and clearly identify, the inductive 
hypothesis, in the form “Assume that P(x) is true for an arbitrary fixed integer k > b.” 

4. State what needs to be proved under the assumption that the inductive hypothesis is true. 
That is, write out what P(k + 1) says. 

5. Prove the statement P(k + 1) making use of the assumption P(x). (Generally, this is the 
most difficult part of a mathematical induction proof. Decide on the most promising 
proof strategy and look ahead to see how to use the induction hypothesis to build your 
proof of the inductive step. Also, be sure that your proof is valid for all integers k with 
k > b, taking care that the proof works for small values of k, including k = b.) 

6. Clearly identify the conclusion of the inductive step, such as by saying “This completes 
the inductive step.” 

7. After completing the basis step and the inductive step, state the conclusion, namely, “By 
mathematical induction, P(n) is true for all integers n with n > b”. 
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You can prove 
a theorem by 
mathematical 
induction 
even if you do 
not have the 
slightest idea 
why it is true! 


Links > 


IH 
Look for the = symbol to 
see where the inductive 
hypothesis is used. 


Readers will find it worthwhile to see how the steps described in the template are completed 
in each of the 14 examples. It will also be useful to follow these guidelines in the solutions of the 
exercises that ask for proofs by mathematical induction. The guidelines that we presented can 
be adapted for each of the variants of mathematical induction that we introduce in the exercises 
and later in this chapter. 


5.1.6 The Good and the Bad of Mathematical Induction 


An important point needs to be made about mathematical induction before we commence a 
study of its use. The good thing about mathematical induction is that it can be used to prove 
a conjecture once it is has been made (and is true). The bad thing about it is that it cannot 
be used to find new theorems. Mathematicians sometimes find proofs by mathematical in- 
duction unsatisfying because they do not provide insights as to why theorems are true. Many 
theorems can be proved in many ways, including by mathematical induction. Proofs of these the- 
orems by methods other than mathematical induction are often preferred because of the insights 
they bring. (See Example 8 and the subsequent remark for an example of this.) 


5.1.7 Examples of Proofs by Mathematical Induction 


Many theorems assert that P(n) is true for all positive integers n, where P(n) is a propositional 
function. Mathematical induction is a technique for proving theorems of this kind. In other 
words, mathematical induction can be used to prove statements of the form Vn P(n), where the 
domain is the set of positive integers. Mathematical induction can be used to prove an extremely 
wide variety of theorems, each of which is a statement of this form. (Remember, many math- 
ematical assertions include an implicit universal quantifier. The statement “if n is a positive 
integer, then n? — nis divisible by 3” is an example of this. Making the implicit universal quan- 
tifier explicit yields the statement “for every positive integer n, n? — n is divisible by 3.”) 

We will use a variety of examples to illustrate how theorems are proved using mathematical 
induction. The theorems we will prove include summation formulae, inequalities, identities for 
combinations of sets, divisibility results, theorems about algorithms, and some other creative 
results. In this section, and in later sections, we will employ mathematical induction to prove 
many other types of results, including the correctness of computer programs and algorithms. 
Mathematical induction can be used to prove a wide variety of theorems both similar to, and 
also quite different from, the examples here. (For proofs by mathematical induction of many 
more interesting and diverse results, see the Handbook of Mathematical Induction by David 
Gunderson [Gu11].) 

There are many opportunities for errors in induction proofs. We will describe some incorrect 
proofs by mathematical induction at the end of this section and in the exercises. To avoid making 
errors in proofs by mathematical induction, try to follow the guidelines for such proofs provided 
previously in Section 5.1.5. 


SEEING WHERE THE INDUCTIVE HYPOTHESIS IS USED To help the reader under- 
stand each of the mathematical induction proofs in this section, we will note where the induc- 
tive hypothesis is used. We indicate this use in three different ways: by explicit mention in the 
text, by inserting the acronym IH (for inductive hypothesis) over an equals sign or a sign for 
an inequality, or by specifying the inductive hypothesis as the reason for a step in a multi-line 
display. 


PROVING SUMMATION FORMULAE We begin by using mathematical induction to prove 
several summation formulae. As we will see, mathematical induction is particularly well suited 
for proving that such formulae are valid. However, summation formulae can be proven in other 
ways. This is not surprising because there are often different ways to prove a theorem. The major 
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disadvantage of using mathematical induction to prove a summation formula is that you cannot 
use it to derive this formula. That is, you must already have the formula before you attempt to 
prove it by mathematical induction. 

Examples 1—4 illustrate how to use mathematical induction to prove summation formulae. 
The first summation formula we will prove by mathematical induction, in Example 1, is a closed 
formula for the sum of the smallest n positive integers. 


Show that if n is a positive integer, then 


(ewig tD 


Solution: Let P(n) be the proposition that the sum of the first n positive integers, 1 +2 + + n = 
ame. isn(n + 1)/2. We must do two things to prove that P(n) is true forn = 1, 2, 3, .... Namely, 
we must show that P(1) is true and that the conditional statement P(k) implies P(k + 1) is true 
for k = 1,2,3,.... 


BASIS STEP: P(1) is true, because 1 = a) 


because 1 is the sum of the first positive integer. The right-hand side is found by substituting 1 
for nin n(n + 1)/2.) 


INDUCTIVE STEP: For the inductive hypothesis we assume that P(k) holds for an arbitrary 
positive integer k. That is, we assume that 


. (The left-hand side of this equation is 1 


1424 k= EED 


Under this assumption, it must be shown that P(k + 1) is true, namely, that 


(k+ DIi(k+1)+1] = (k + 1)(k + 2) 


L+24+--+k+(kt)= 
(k +1) 5 5 


is also true. 

We now look ahead to see how we might be able to prove that P(k + 1) holds under the 
assumption that P(x) is true. We observe that the summation in the left-hand side of P(k + 1) is 
k + 1 more than the summation in the left-hand side of P(x). Our strategy will be to add k + 1 to 
both sides of the equation in P(k) and simplify the result algebraically to complete the inductive 
step. 

We now return to the proof of the inductive step. When we add k + 1 to both sides of the 
equation in P(k), we obtain 


Lt+2+-+¢k+(k+D= 


= RHD at 
2 

_kk+1)+2&k+1) 

—— 


_ kt DK+2) 
ee. 


This last equation shows that P(A + 1) is true under the assumption that P(k) is true. This com- 
pletes the inductive step. 


We have completed the basis step and the inductive step, so by mathematical induction we 
know that P(n) is true for all positive integers n. That is, we have proven that 1 +2 + = +n = 
n(n + 1)/2 for all positive integers n. < 
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EXAMPLE 2 


As we noted, mathematical induction is not a tool for finding theorems about all positive 
integers. Rather, it is a proof method for proving such results once they are conjectured. In 
Example 2, using mathematical induction to prove a summation formula, we will both formulate 
and then prove a conjecture. 


Conjecture a formula for the sum of the first n positive odd integers. Then prove your conjecture 
using mathematical induction. 


Solution: The sums of the first n positive odd integers for n = 1, 2, 3, 4, 5 are 


f=, 1+3=4, 1+3+5=9, 
1+3+5+7=16, 1+3+5+7+9=25. 


From these values it is reasonable to conjecture that the sum of the first n positive odd integers 
isn’, that is, 1 +3 + 5+ = + (2n — 1) =n’. We need a method to prove that this conjecture is 
correct, if in fact it is. 

Let P(n) denote the proposition that the sum of the first n odd positive integers is n?. Our 
conjecture is that P(n) is true for all positive integers n. To use mathematical induction to prove 
this conjecture, we must first complete the basis step; that is, we must show that P(1) is true. 
Then we must carry out the inductive step; that is, we must show that P(k + 1) is true when P(k) 
is assumed to be true. We now attempt to complete these two steps. 


BASIS STEP: P(A) states that the sum of the first one odd positive integer is 17. This is true 
because the sum of the first odd positive integer is 1. The basis step is complete. 


INDUCTIVE STEP: To complete the inductive step we must show that the proposition 
P(k) > P(k + 1) is true for every positive integer k. To do this, we first assume the inductive 
hypothesis. The inductive hypothesis is the statement that P(x) is true for an arbitrary positive 
integer k, that is, 


14+3454+--+(2k-D=Rk. 


(Note that the kth odd positive integer is (2k — 1), because this integer is obtained by adding 2 
a total of k — 1 times to 1.) 

To show that Vk(P(k) > P(k + 1)) is true, we must show that if P(k) is true (the inductive 
hypothesis), then P(k + 1) is true. Note that P(k + 1) is the statement that 


1434544 (2k-1)4+(2k4+ 1) =(k4+1¥. 


Before we complete the inductive step, we will take a time out to figure out a strategy. At 
this stage of a mathematical induction proof it is time to look for a way to use the inductive 
hypothesis to show that P(k + 1) is true. Here we note that 1 +3 +5 + --- + (2k — 1) + (2k + 1) 
is the sum of its first k terms 1 +3 +5 +--- + (2k — 1) and its last term 2k — 1. So, we can use 
our inductive hypothesis to replace 1 + 3 +5 +++: + (2k — 1) by k. 

We now return to our proof. We find that 


EES EEE E E = [199 4:4 Ok= HOES 
= R24 (2k +1) 
=k +2k+1 
=e i), 


This shows that P(k + 1) follows from P(k). Note that we used the inductive hypothesis P(x) in 
the second equality to replace the sum of the first k odd positive integers by k?. 


EXAMPLE 3 


EXAMPLE 4 
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We have now completed both the basis step and the inductive step. That is, we have shown 
that P(1) is true and the conditional statement P(k) > P(k + 1) is true for all positive integers k. 
Consequently, by the principle of mathematical induction we can conclude that P(n) is true 
for all positive integers n. That is, we know that 1 +3 +5 +++ + (2n — 1) = n? for all positive 
integers n. < 


Use mathematical induction to show that 
LH 242? $427 = QM n] 
for all nonnegative integers n. 


Solution: Let P(n) be the proposition that 1 + 2 + 27 + --» +2” = 2"+! — 1 for the integer n. 
BASIS STEP: P(O) is true because 2° = 1 = 2! — 1. This completes the basis step. 
INDUCTIVE STEP: For the inductive hypothesis, we assume that P(x) is true for an arbitrary 
nonnegative integer k. That is, we assume that 


142427 40-42% = 21-1. 


To carry out the inductive step using this assumption, we must show that when we assume 
that P(k) is true, then P(k + 1) is also true. That is, we must show that 


142427 4-6 42% 421 = 21 _ pak? _ | 
assuming the inductive hypothesis P(k). Under the assumption of P(k), we see that 
LH2427 40-42% 421 = (142427 45-424) 42"! 


= (kt = 1) 4 Qk+1 
= 2.21] 


= Qk+2 =]. 


Note that we used the inductive hypothesis in the second equation in this string of equalities to 
replace 1 +2 +2? + --- +2% by 24+! — 1. We have completed the inductive step. 


Because we have completed the basis step and the inductive step, by mathematical induction 
we know that P(n) is true for all nonnegative integers n. That is, 1 +2 + = +2” = 2”+ — 1 for 
all nonnegative integers n. < 


The formula given in Example 3 is a special case of a general result for the sum of terms 
of a geometric progression (Theorem 1 in Section 2.4). We will use mathematical induction to 
provide an alternative proof of this formula. 


Sums of Geometric Progressions Use mathematical induction to prove this formula for the 
sum of a finite number of terms of a geometric progression with initial term a and common 
ratio r: 


n 
Yi ar =a+ar+ar + "+ ar" = 
j=0 


n+l _ 


ar a 


when r Æ 1, 
r-1l 


where n is a nonnegative integer. 
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Solution: To prove this formula using mathematical induction, let P(n) be the statement that the 
sum of the first n + 1 terms of a geometric progression in this formula is correct. 


BASIS STEP: P(O) is true, because 


-a ar-a ar-1) 
= = = 


r-1 r-1l r-1 


INDUCTIVE STEP: The inductive hypothesis is the statement that P(k) is true, where k is an 
arbitrary nonnegative integer. That is, P(k) is the statement that 


2 p_ art! —a 
a+ar+ar +: +ar = 


r-1l 


To complete the inductive step we must show that if P(x) is true, then P(k + 1) is also true. To 
show that this is the case, we first add ar*! to both sides of the equality asserted by P(k). We 
find that 

k+l 


Har’ —a 


I 
2 . 
adtartar ++ + arf +ar*! = + arkt!, 


r-1 


Rewriting the right-hand side of this equation shows that 


art! -a pee art! -a art? — art! 
r-1l r-l r-1l 
= art? — a 
r-1l 


Combining these last two equations gives 


ar -a 


a+ar+ar + + ar +art! = 
r- 


This shows that if the inductive hypothesis P(x) is true, then P(k + 1) must also be true. This 
completes the inductive argument. 


We have completed the basis step and the inductive step, so by mathematical induction P(n) 
is true for all nonnegative integers n. This shows that the formula for the sum of the terms of a 
geometric series is correct. < 


As previously mentioned, the formula in Example 3 is the case of the formula in 
Example 4 with a = 1 and r = 2. The reader should verify that putting these values for a and r 
into the general formula gives the same formula as in Example 3. 


PROVING INEQUALITIES Mathematical induction can be used to prove a variety of in- 


equalities that hold for all positive integers greater than a particular positive integer, as 
Examples 5-7 illustrate. 


Use mathematical induction to prove the inequality 
n<?” 
for all positive integers n. 


Solution: Let P(n) be the proposition that n < 2”. 
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Extra » BASIS STEP: P(1) is true, because 1 < 2! = 2. This completes the basis step. 


Examples 


EXAMPLE 6 


EXAMPLE 7 


INDUCTIVE STEP: We first assume the inductive hypothesis that P(x) is true for an arbitrary 
positive integer k. That is, the inductive hypothesis P(k) is the statement that k < 2. To complete 
the inductive step, we need to show that if P(x) is true, then P(k + 1), which is the statement 
that k+ 1 < 2‘*! is true. That is, we need to show that if k < 2‘, then k + 1 < 2%+!. To show 
that this conditional statement is true for the positive integer k, we first add 1 to both sides of 
k < 2*, and then note that 1 < 2“. This tells us that 


IH 

kt1<2%41<2%42%=2.2% = 21, 
This shows that P(k + 1) is true, namely, that k + 1 < 2‘+!, based on the assumption that P(k) 
is true. The induction step is complete. 


Therefore, because we have completed both the basis step and the inductive step, by 
the principle of mathematical induction we have shown that n < 2” is true for all positive 
integers n. < 


Use mathematical induction to prove that 2” < n! for every integer n with n > 4. (Note that this 
inequality is false for n = 1, 2, and 3.) 
Solution: Let P(n) be the proposition that 2” < n!. 


BASIS STEP: To prove the inequality for n > 4 requires that the basis step be P(4). Note that 
P(A) is true, because 24 = 16 < 24 = 4! 


INDUCTIVE STEP: For the inductive step, we assume that P(x) is true for an arbitrary integer k 
with k > 4. That is, we assume that 2% < k! for the positive integer k with k > 4. We must show 
that under this hypothesis, P(k + 1) is also true. That is, we must show that if 2% < k! for an 
arbitrary positive integer k where k > 4, then 2‘t! < (k + 1)!. We have 


ae = 2. 2k by definition of exponent 


IH 

<2-k! by the inductive hypothesis 

<(kK+1)k!  because2<k+1 

=(k+1)! by definition of factorial function. 
This shows that P(k + 1) is true when P(k) is true. This completes the inductive step of the 
proof. 


We have completed the basis step and the inductive step. Hence, by mathematical induction 
P(n) is true for all integers n with n > 4. That is, we have proved that 2” < n! is true for all 
integers n with n > 4. < 


An important inequality for the sum of the reciprocals of a set of positive integers will be 
proved in Example 7. 


An Inequality for Harmonic Numbers The harmonic numbers H, j= 1, 2,3,..., are 
defined by 


1 1 1 
He=1+itst¢--4+5. 
te aa 
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For instance, 


a 


+ 


ppl 
2 3 4 12’ 


Use mathematical induction to show that 
n 
Hy, > 1+5, 
2 2 
whenever n is a nonnegative integer. 


Solution: To carry out the proof, let P(n) be the proposition that H,, > 1 + ` 


BASIS STEP: P(Q) is true, because Hy) = H =1 2> 1+ >. 


INDUCTIVE STEP: The inductive hypothesis is the statement that P(k) is true, that is, 
Ay >1+ where k is an arbitrary nonnegative integer. We must show that if P(A) is true, 


then PEI which sinew that aS 4 2 


hypothesis, it follows that 


, is also true. So, assuming the inductive 


1 1 1 1 1 
Hyn =Ltotioteet m + Tar +e + Fi by the definition of harmonic 
2 3 2 2k +1 2 nibe 
1 1 
= Ay. + Tes P AR by the definition of 2th harmonic 
ar +1 2 number 
H k 1 1 , ! , 
>(1+ 5 + ad paa JT by the inductive hypothesis 
k cr 1 k 
>(1l4+=)4+2*- ae] because there are 2“ terms 
2 2 each > 1/2*+! 
k 1 . 
>(1+ z T 5 canceling a common factor of 
2* in second term 
k+1 
=1+ 
2 


This establishes the inductive step of the proof. 


We have completed the basis step and the inductive step. Thus, by mathematical induction 
P(n) is true for all nonnegative integers n. That is, the inequality H,, > 1+ > for the harmonic 
numbers holds for all nonnegative integers n. < 


Remark: The inequality established here shows that the harmonic series 


ee ae ae to ha 
2 3 n 


is a divergent infinite series. This is an important example in the study of infinite series. 


PROVING DIVISIBILITY RESULTS Mathematical induction can be used to prove divisi- 
bility results about integers. Although such results are often easier to prove using basic results in 
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number theory, it is instructive to see how to prove such results using mathematical induction, 
as Examples 8 and 9 illustrate. 


Use mathematical induction to prove that n> —n is divisible by 3 whenever n is a posi- 
tive integer. (Note that this is the statement with p = 3 of Fermat’s little theorem, which is 
Theorem 3 of Section 4.4.) 


Solution: To construct the proof, let P(n) denote the proposition: “n? 


BASIS STEP: The statement P(1) is true because 1° — 1 = 0 is divisible by 3. This completes 
the basis step. 


— nis divisible by 3.” 


INDUCTIVE STEP: For the inductive hypothesis we assume that P(A) is true; that is, we assume 
that k? — k is divisible by 3 for an arbitrary positive integer k. To complete the inductive step, we 
must show that when we assume the inductive hypothesis, it follows that P(k + 1), the statement 
that (k + 1} — (k + 1) is divisible by 3, is also true. That is, we must show that (k + 1)? — (k + 1) 
is divisible by 3. Note that 


(K+ 1% —(k4+1) = (P 43k? +3k4+-1)-(K +1) 
= (K —k) +3(k? +h). 


Using the inductive hypothesis, we conclude that the first term k? — k is divisible by 3. The 
second term is divisible by 3 because it is 3 times an integer. So, by part (i) of Theorem 1 in 
Section 4.1, we know that (k + 1)° — (k + 1) is also divisible by 3. This completes the inductive 
step. 


Because we have completed both the basis step and the inductive step, by the prin- 
ciple of mathematical induction we know that n? —n is divisible by 3 whenever n is a 
positive integer. < 


Remark: We have included Example 8 as an illustration how a divisibility result can be proved 
by mathematical induction. However, there are simpler proofs. For example, we can prove that 
n> — nis divisible by 3 for all positive integers n using the factorization n? — n = n(n? — 1) = 
n(n— 1)(n+ 1) = (n — l)n(n + 1). Hence, n? — 1 is divisible by 3 because it is the product of 
three consecutive integers, one of which is divisible by 3. 


Example 9 presents a more challenging proof by mathematical induction of a divisibility 
result. 


Use mathematical induction to prove that 7”** + 87”*! is divisible by 57 for every nonnegative 
integer n. 


Solution: To construct the proof, let P(n) denote the proposition: “7"*? + 8?”+! is divisible 
by 57.” 


BASIS STEP: To complete the basis step, we must show that P(O) is true, because we want 
to prove that P(n) is true for every nonnegative integer n. We see that P(0) is true because 
70+? + 820+! = 7? + 8! = 57 is divisible by 57. This completes the basis step. 


INDUCTIVE STEP: For the inductive hypothesis we assume that P(x) is true for an arbitrary 
nonnegative integer k; that is, we assume that 7+? + 87+! is divisible by 57. To complete the 
inductive step, we must show that when we assume that the inductive hypothesis P(k) is true, 
then P(k + 1), the statement that 7+)+? + 824+)+! is divisible by 57, is also true. 
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EXAMPLE 11 


The difficult part of the proof is to see how to use the inductive hypothesis. To take advan- 
tage of the inductive hypothesis, we use these steps: 


TOHID 4 QAUk+D +1 — 7k+3 4 Q2k+3 
= 7 -H2 4 82 . 82+! 
= 7. 72 + 64 . 82%! 
= (7+2 + 8%+!) + 57 . 8%+!, 


We can now use the inductive hypothesis, which states that 74+? + 8%+! is divisible by 57. 
We will use parts (i) and (ii) of Theorem 1 in Section 4.1. By part (ii) of this theorem, and the 
inductive hypothesis, we conclude that the first term in this last sum, 7(7%+2 + 8%+!), is divisible 
by 57. By part (ii) of this theorem, the second term in this sum, 57 - 8%+1_ is divisible by 57. 
Hence, by part (i) of this theorem, we conclude that 7(7%+? + 874+!) + 57 . 92+! = 7k+3 4 g2k+3 
is divisible by 57. This completes the inductive step. 


Because we have completed both the basis step and the inductive step, by the principle 
of mathematical induction we know that 7"*? + 8"*! is divisible by 57 for every nonnegative 
integer n. < 


PROVING RESULTS ABOUT SETS Mathematical induction can be used to prove many re- 
sults about sets. In particular, in Example 10 we prove a formula for the number of subsets of a 
finite set and in Example 11 we establish a set identity. 


The Number of Subsets of a Finite Set Use mathematical induction to show that if S is a 
finite set with n elements, where n is a nonnegative integer, then S has 2” subsets. (We will prove 
this result directly in several ways in Chapter 6.) 


Solution: Let P(n) be the proposition that a set with n elements has 2” subsets. 


BASIS STEP: P(Q) is true, because a set with zero elements, the empty set, has exactly 2° =1 
subset, namely, itself. 


INDUCTIVE STEP: For the inductive hypothesis we assume that P(x) is true for an arbitrary 
nonnegative integer k, that is, we assume that every set with k elements has 2* subsets. It must 
be shown that under this assumption, P(k + 1), which is the statement that every set with k + 1 
elements has 2‘+! subsets, must also be true. To show this, let T be a set with k + 1 elements. 
Then, it is possible to write T = SU {a}, where a is one of the elements of T and S = T — {a} 
(and hence || = k). The subsets of T can be obtained in the following way. For each subset X 
of S there are exactly two subsets of T, namely, X and X U {a}. (This is illustrated in Figure 3.) 
These constitute all the subsets of T and are all distinct. We now use the inductive hypothesis 
to conclude that S has 2* subsets, because it has k elements. We also know that there are two 
subsets of T for each subset of S. Therefore, there are 2 - 2* = 24+! subsets of T. This finishes 
the inductive argument. 


Because we have completed the basis step and the inductive step, by mathematical induction 
it follows that P(n) is true for all nonnegative integers n. That is, we have proved that a set with n 
elements has 2” subsets whenever n is a nonnegative integer. < 


Use mathematical induction to prove the following generalization of one of De Morgan’s laws: 


whenever A}, A>, ..., A, are subsets of a universal set U and n > 2. 
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FIGURE 3 Generating subsets of a set with k + 1 elements. Here T = S u {a}. 


Solution: Let P(n) be the identity for n sets. 


BASIS STEP: The statement P(2) asserts that A, N A, = A, UA). This is one of De Morgan’s 
laws; it was proved in Example 11 of Section 2.2. 


INDUCTIVE STEP: The inductive hypothesis is the statement that P(k) is true, where k is an 
arbitrary integer with k > 2; that is, it is the statement that 


whenever A}, A>, ..., A, are subsets of the universal set U. To carry out the inductive step, we 
need to show that this assumption implies that P(k + 1) is true. That is, we need to show that 
if this equality holds for every collection of k subsets of U, then it must also hold for every 
collection of k + 1 subsets of U. Suppose that A}, A>, ..., Ap Az, are subsets of U. When the 
inductive hypothesis is assumed to hold, it follows that 


A ) NAg by the definition of intersection 


by De Morgan’s law (where the two sets are (i. 14 and A;,)) 


II 
k 
iD 
œ 
Se” 
(= 
> 
: 


k 
IH — 
= ( U 5) U AÁ} by the inductive hypothesis 


= . by the definition of union. 


This completes the inductive step. 
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Because we have completed both the basis step and the inductive step, by mathematical 
induction we know that P(n) is true whenever n is a positive integer, n > 2. That is, we know 
that 


whenever A,, A,, ...,A,, are subsets of a universal set U and n > 2. < 


PROVING RESULTS ABOUT ALGORITHMS Next, we provide an example (somewhat 
more difficult than previous examples) that illustrates one of many ways mathematical induction 
is used in the study of algorithms. We will show how mathematical induction can be used to 
prove that a greedy algorithm we introduced in Section 3.1 always yields an optimal solution. 


Recall the algorithm for scheduling talks discussed in Example 7 of Section 3.1. The input to 
this algorithm is a group of m proposed talks with preset starting and ending times. The goal is 
to schedule as many of these lectures as possible in the main lecture hall so that no two talks 
overlap. Suppose that talk £, begins at time s; and ends at time ¢. (No two lectures can proceed 
in the main lecture hall at the same time, but a lecture in this hall can begin at the same time 
another one ends.) 

Without loss of generality, we assume that the talks are listed in order of nondecreasing 
ending time, so that e} < e < --- < €m- The greedy algorithm proceeds by selecting at each 
stage a talk with the earliest ending time among all those talks that begin no sooner than when 
the last talk scheduled in the main lecture hall has ended. Note that a talk with the earliest end 
time is always selected first by the algorithm. We will show that this greedy algorithm is optimal 
in the sense that it always schedules the most talks possible in the main lecture hall. To prove 
the optimality of this algorithm we use mathematical induction on the variable n, the number 
of talks scheduled by the algorithm. We let P(n) be the proposition that if the greedy algorithm 
schedules n talks in the main lecture hall, then it is not possible to schedule more than n talks 
in this hall. 


BASIS STEP: Suppose that the greedy algorithm managed to schedule just one talk, tį, in 
the main lecture hall. This means that no other talk can start at or after e}, the end time of t. 
Otherwise, the first such talk we come to as we go through the talks in order of nondecreasing 
end times could be added. Hence, at time e, each of the remaining talks needs to use the main 
lecture hall because they all start before e, and end after e,. It follows that no two talks can be 
scheduled because both need to use the main lecture hall at time e,. This shows that P(1) is true 
and completes the basis step. 


INDUCTIVE STEP: The inductive hypothesis is that P(k) is true, where k is an arbitrary positive 
integer, that is, that the greedy algorithm always schedules the most possible talks when it selects 
k talks, where k is a positive integer, given any set of talks, no matter how many. We must show 
that P(k + 1) follows from the assumption that P(x) is true, that is, we must show that under 
the assumption of P(x), the greedy algorithm always schedules the most possible talks when it 
selects k + 1 talks. 

Now suppose that the greedy algorithm has selected k + 1 talks. Our first step in completing 
the inductive step is to show there is a schedule including the most talks possible that contains 
talk ¢,, a talk with the earliest end time. This is easy to see because a schedule that begins 
with the talk f, in the list, where i > 1, can be changed so that talk t, replaces talk t;. To see 
this, note that because e} < e, all talks that were scheduled to follow talk t; can still be scheduled. 

Once we included talk t,, scheduling the talks so that as many as possible are scheduled 
is reduced to scheduling as many talks as possible that begin at or after time e,. So, if we 
have scheduled as many talks as possible, the schedule of talks other than talk ¢, is an optimal 
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schedule of the original talks that begin once talk t; has ended. Because the greedy algorithm 
schedules k talks when it creates this schedule, we can apply the inductive hypothesis to con- 
clude that it has scheduled the most possible talks. It follows that the greedy algorithm has sched- 
uled the most possible talks, k + 1, when it produced a schedule with k + 1 talks, so P(k + 1) is 
true. This completes the inductive step. 


We have completed the basis step and the inductive step. So, by mathematical induction we 
know that P(n) is true for all positive integers n. This completes the proof of optimality. That is, 
we have proved that when the greedy algorithm schedules n talks, when n is a positive integer, 
then it is not possible to schedule more than n talks. < 


CREATIVE USES OF MATHEMATICAL INDUCTION Mathematical induction can often 
be used in unexpected ways. We will illustrate two particularly clever uses of mathematical 
induction here, the first relating to survivors in a pie fight and the second relating to tilings with 
regular triominoes of checkerboards with one square missing. 


Odd Pie Fights An odd number of people stand in a yard at mutually distinct distances. At 
the same time each person throws a pie at their nearest neighbor, hitting this person. Use math- 
ematical induction to show that there is at least one survivor, that is, at least one person who is 
not hit by a pie. (This problem was introduced by Carmony [Ca79]. Note that this result is false 
when there are an even number of people; see Exercise 77.) 


Solution: Let P(n) be the statement that there is a survivor whenever 2n + 1 people stand in 
a yard at distinct mutual distances and each person throws a pie at their nearest neighbor. To 
prove this result, we will show that P(n) is true for all positive integers n. This follows because 
as n runs through all positive integers, 2n + 1 runs through all odd integers greater than or equal 
to 3. Note that one person cannot engage in a pie fight because there is no one else to throw the 
pie at. 


BASIS STEP: When n = 1, there are 2n + 1 = 3 people in the pie fight. Of the three people, 
suppose that the closest pair are A and B, and C is the third person. Because distances between 
pairs of people are different, the distance between A and C and the distance between B and C 
are both different from, and greater than, the distance between A and B. It follows that A and B 
throw pies at each other, while C throws a pie at either A or B, whichever is closer. Hence, C is 
not hit by a pie. This shows that at least one of the three people is not hit by a pie, completing 
the basis step. 


INDUCTIVE STEP: For the inductive step, assume that P(x) is true for an arbitrary odd integer 
k with k > 3. That is, assume that there is at least one survivor whenever 2k + 1 people stand in 
a yard at distinct mutual distances and each throws a pie at their nearest neighbor. We must show 
that if the inductive hypothesis P(k) is true, then P(k + 1), the statement that there is at least one 
survivor whenever 2(k + 1) + 1 = 2k + 3 people stand in a yard at distinct mutual distances and 
each throws a pie at their nearest neighbor, is also true. 

So suppose that we have 2(k + 1)+ 1 = 2k +3 people in a yard with distinct distances 
between pairs of people. Let A and B be the closest pair of people in this group of 2k + 3 people. 
When each person throws a pie at the nearest person, A and B throw pies at each other. We have 
two cases to consider, (i) when someone else throws a pie at either A or B and (ii) when no one 
else throws a pie at either A or B. 


Case (i): Because A and B throw pies at each other and someone else throws a pie at either A 
and B, at least three pies are thrown at A and B, and at most (2k + 3) — 3 = 2k pies are thrown 
at the remaining 2k + 1 people. This guarantees that at least one person is a survivor, for if each 
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EXAMPLE 14 


FIGURE4 A 
right triomino. 


of these 2k + 1 people was hit by at least one pie, a total of at least 2k + 1 pies would have to be 
thrown at them. (The reasoning used in this last step is an example of the pigeonhole principle 
discussed further in Section 6.2.) 


Case (ii): No one else throws a pie at either A and B. Besides A and B, there are 2k + 1 people. 
Because the distances between pairs of these people are all different, we can use the induc- 
tive hypothesis to conclude that there is at least one survivor S when these 2k + 1 people each 
throws a pie at their nearest neighbor. Furthermore, S is also not hit by either the pie thrown 
by A or the pie thrown by B because A and B throw their pies at each other, so S is a survivor 
because S is not hit by any of the pies thrown by these 2k + 3 people. 


We have completed both the basis step and the inductive step, using a proof by cases. So by 
mathematical induction it follows that P(n) is true for all positive integers n. We conclude that 
whenever an odd number of people located in a yard at distinct mutual distances each throws a 
pie at their nearest neighbor, there is at least one survivor. < 


In Section 1.8 we discussed the tiling of checkerboards by polyominoes. Example 14 illus- 
trates how mathematical induction can be used to prove a result about covering checkerboards 
with right triominoes, pieces shaped like the letter “L.” 


Let n be a positive integer. Show that every 2” x 2” checkerboard with one square removed can 
be tiled using right triominoes, where these pieces cover three squares at a time, as shown in 
Figure 4. 


Solution: Let P(n) be the proposition that every 2” x 2” checkerboard with one square removed 
can be tiled using right triominoes. We can use mathematical induction to prove that P(n) is 
true for all positive integers n. 


BASIS STEP: P(1) is true, because each of the four 2 x 2 checkerboards with one square re- 
moved can be tiled using one right triomino, as shown in Figure 5. 


FIGURE 5 Tiling 2 x 2 checkerboards with one square removed. 


INDUCTIVE STEP: The inductive hypothesis is the assumption that P(X) is true for the positive 
integer k; that is, it is the assumption that every 2% x 2% checkerboard with one square removed 
can be tiled using right triominoes. It must be shown that under the assumption of the inductive 
hypothesis, P(A + 1) must also be true; that is, any 21 x 2'+1 Checkerboard with one square 
removed can be tiled using right triominoes. 

To see this, consider a 2%*+! x 2! checkerboard with one square removed. Split this 
checkerboard into four checkerboards of size 2% x 2*, by dividing it in half in both directions. 
This is illustrated in Figure 6. No square has been removed from three of these four checker- 
boards. The fourth 2% x 2 checkerboard has one square removed, so we now use the inductive 
hypothesis to conclude that it can be covered by right triominoes. Now temporarily remove the 
square from each of the other three 2" x 2* checkerboards that has the center of the original, 
larger checkerboard as one of its corners, as shown in Figure 7. By the inductive hypothesis, 
each of these three 2% x 2* checkerboards with a square removed can be tiled by right triomi- 
noes. Furthermore, the three squares that were temporarily removed can be covered by one right 
triomino. Hence, the entire 24+! x 24+! checkerboard can be tiled with right triominoes. 


Consult Common Errors 
in Discrete Mathematics 
on this book’s website 

for more basic mistakes. 
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FIGURE 6 Dividing a FIGURE7 Tiling the 
2'+1 x 2+! checkerboard into 24+1 x 2k+1 checkerboard 
four 2" x 2 checkerboards. with one square removed. 


We have completed the basis step and the inductive step. Therefore, by mathematical in- 
duction P(n) is true for all positive integers n. This shows that we can tile every 2” x 2” checker- 
board, where n is a positive integer, with one square removed, using right triominoes. < 


5.1.8 Mistaken Proofs By Mathematical Induction 


As with every proof method, there are many opportunities for making errors when using mathe- 
matical induction. Many well-known mistaken, and often entertaining, proofs by mathematical 
induction of clearly false statements have been devised, as exemplified by Example 15 and 
Exercises 49-51. Often, it is not easy to find where the error in reasoning occurs in such mis- 
taken proofs. 

To uncover errors in proofs by mathematical induction, remember that in every such proof, 
both the basis step and the inductive step must be done correctly. Not completing the basis step 
in a supposed proof by mathematical induction can lead to mistaken proofs of clearly ridiculous 
statements such as “n = n + 1 whenever n is a positive integer.” (We leave it to the reader to 
show that it is easy to construct a correct inductive step in an attempted proof of this statement.) 
Locating the error in a faulty proof by mathematical induction, as Example 15 illustrates, can 
be quite tricky, especially when the error is hidden in the basis step. 


Find the error in this “proof” of the clearly false claim that every set of lines in the plane, no 
two of which are parallel, meet in a common point. 


“Proof:” Let P(n) be the statement that every set of n lines in the plane, no two of which are 
parallel, meet in a common point. We will attempt to prove that P(n) is true for all positive 
integers n > 2. 


BASIS STEP: The statement P(2) is true because any two lines in the plane that are not parallel 
meet in a common point (by the definition of parallel lines). 


INDUCTIVE STEP: The inductive hypothesis is the statement that P(x) is true for the positive 
integer k, that is, it is the assumption that every set of k lines in the plane, no two of which are 
parallel, meet in a common point. To complete the inductive step, we must show that if P(x) is 
true, then P(k + 1) must also be true. That is, we must show that if every set of k lines in the 
plane, no two of which are parallel, meet in a common point, then every set of k + 1 lines in the 
plane, no two of which are parallel, meet in a common point. So, consider a set of k + 1 distinct 
lines in the plane. By the inductive hypothesis, the first k of these lines meet in a common point 
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pı. Moreover, by the inductive hypothesis, the last k of these lines meet in a common point 
P2. We will show that p, and p, must be the same point. If p, and p, were different points, all 
lines containing both of them must be the same line because two points determine a line. This 
contradicts our assumption that all these lines are distinct. Thus, p, and p, are the same point. 
We conclude that the point p; = p, lies on all k + 1 lines. We have shown that P(k + 1) is true 
assuming that P(k) is true. That is, we have shown that if we assume that every k, k > 2, distinct 
lines meet in a common point, then every k + 1 distinct lines meet in a common point. This 
completes the inductive step. 


We have completed the basis step and the inductive step, and supposedly we have a correct 
proof by mathematical induction. 


Solution: Examining this supposed proof by mathematical induction it appears that everything 
is in order. However, there is an error, as there must be. The error is rather subtle. Carefully 
looking at the inductive step shows that this step requires that k > 3. We cannot show that P(2) 
implies P(3). When k = 2, our goal is to show that every three distinct lines meet in a common 
point. The first two lines must meet in a common point p, and the last two lines must meet in 
a common point p,. But in this case, p, and p, do not have to be the same, because only the 


second line is common to both sets of lines. Here is where the inductive step fails. < 


Exercises 


1. There are infinitely many stations on a train route. Sup- 
pose that the train stops at the first station and suppose 
that if the train stops at a station, then it stops at the next 
station. Show that the train stops at all stations. 


2. Suppose that you know that a golfer plays the first hole of 
a golf course with an infinite number of holes and that if 
this golfer plays one hole, then the golfer goes on to play 
the next hole. Prove that this golfer plays every hole on 


c) What is the inductive hypothesis of a proof that P(n) 
is true for all positive integers n? 

d) What do you need to prove in the inductive step of a 
proof that P(n) is true for all positive integers n? 

e) Complete the inductive step of a proof that P(n) is 
true for all positive integers n, identifying where you 
use the inductive hypothesis. 

f) Explain why these steps show that this formula is true 
whenever n is a positive integer. 


th : 
ae ee 5. Prove that 12 + 32 + 5? +--+ 2n + 1} = (n + 1) 2n + 
pa e a. h uction P ee oO. to parea 1)@n + 3)/3 whenever n is a nonnegative integer. 
mation formulae. Be sure to identify where you use the in- 6. Prove that 1-11+2-21+--+n-n!l=(n+)!-1 
ductive hypothesis. : paei 
whenever n is a positive integer. 
3. Let P(n) be the statement that 1° +2? + + + n? = n(n + 7. Prove that 3+3-54+3 -5+ +3: 5"=3(5"+!— 1)/4 
1)(2n + 1)/6 for the positive integer n. whenever n is a nonnegative integer. 
a) What is the statement P(1)? 8. Prove that 2—-2-74+2-7—-+-4+2(-7)"=(1- 
b) Show that P(1) is true, completing the basis step of a (—7)"*!)/4 whenever n is a nonnegative integer. 
proof that P(n) is true for all positive integers n. 9. a) Find a formula for the sum of the first n even positive 
c) What is the inductive hypothesis of a proof that P(n) integers. 
is true for all positive integers n? b) Prove the formula that you conjectured in part (a). 
d) What do you need to prove in the inductive step of a 10. a) Find a formula for 
proof that P(n) is true for all positive integers n? 1 j; 1 poet 1 
e) Complete the inductive step of a proof that P(n) is 1-2 2-3 n(n + 1) 
true for all positive integers n, identifying where you by examining the values of this expression for small 
use the inductive hypothesis. values of n. 
f) Explain why these steps show that this formula is true b) Prove the formula you conjectured in part (a). 
whenever n is a positive integer. 11. a) Find a formula for 


4. Let P(n) be the statement that 1? + 2? +- +73 = (n(n + 
1)/2)* for the positive integer n. 
a) What is the statement P(1)? 
b) Show that P(1) is true, completing the basis step of 
the proof of P(n) for all positive integers n. 


Te PERY 
2 4 8 2n 
by examining the values of this expression for small 
values of n. 
b) Prove the formula you conjectured in part (a). 


12. 


13. 


14. 


15. 


16. 


17. 


Prove that 


n 


Da 


j=0 


whenever n is a nonnegative integer. 
Prove that 1? -— 2? +3? -— +. +(-1)"'n? =(-1)""! 
n(n + 1)/2 whenever n is a positive integer. 


Prove that for every positive integer n, Ð% kak = 
(n—1)2"*! 4.2. 
Prove that for every positive integer n, 
1-242-34+--4+n(n4+ 1) = n(n + 1)(n + 2)/3. 
Prove that for every positive integer n, 
1-2-342-3-44+--+n(n+ l(n+ 2) 
=n(nt+ 1)(n + 2)(n + 3)/4. 


Prove that 2. -J = n(n + 1I)Qn+ 1)Bn? +3n—1)/30 
whenever n is a positive integer. 


Use mathematical induction to prove the inequalities in Exer- 
cises 18-30. 


18. 


19. 


Let P(n) be the statement that n! < n”, where n is an in- 
teger greater than 1. 


a) What is the statement P(2)? 

b) Show that P(2) is true, completing the basis step of a 
proof by mathematical induction that P(n) is true for 
all integers n greater than 1. 

c) What is the inductive hypothesis of a proof by math- 
ematical induction that P(n) is true for all integers n 
greater than 1? 

d) What do you need to prove in the inductive step of a 
proof by mathematical induction that P(n) is true for 
all integers n greater than 1? 

e) Complete the inductive step of a proof by mathemati- 
cal induction that P(n) is true for all integers n greater 
than 1. 

f) Explain why these steps show that this inequality is 
true whenever n is an integer greater than 1. 


Let P(n) be the statement that 


1... 1 1 1 
lt+o4+- t <2, 

4 9 n? n 
where n is an integer greater than 1. 


a) What is the statement P(2)? 

b) Show that P(2) is true, completing the basis step of a 
proof by mathematical induction that P(n) is true for 
all integers n greater than 1. 

c) What is the inductive hypothesis of a proof by math- 
ematical induction that P(n) is true for all integers n 
greater than 1? 

d) What do you need to prove in the inductive step of a 
proof by mathematical induction that P(n) is true for 
all integers n greater than 1? 

e) Complete the inductive step of a proof by mathemati- 
cal induction that P(n) is true for all integers n greater 
than 1. 

f) Explain why these steps show that this inequality is 
true whenever n is an integer greater than 1. 


20. 
21. 
22. 


23. 


24. 


* 25. 


* 26. 


* 27. 


28. 
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Prove that 3”< n! if n is an integer greater than 6. 

Prove that 2” > n? if n is an integer greater than 4. 

For which nonnegative integers n is n? < n!? Prove your 
answer. 

For which nonnegative integers n is 2n + 3 < 2”? Prove 
your answer. 

Prove that 1/(2n) < [1-3-5 -+ - (2n — 1)]/(2 -4 - 
2n) whenever n is a positive integer. 

Prove that if h > —1, then 1 +nh < (1 + h)” for all non- 
negative integers n. This is called Bernoulli’s inequality. 
Suppose that a and b are real numbers with 0 < b < a. 
Prove that if n is a positive integer, then a” — b” < 
na™! (a — b). 

Prove that for every positive integer n, 


Pe pis spisa] n+1-—1). 
avi yn 


Prove that n? — 7n + 12 is nonnegative whenever n is an 
integer with n > 3. 


In Exercises 29 and 30, H,, denotes the nth harmonic number. 


* 29, 


* 30. 


Prove that H,, <1-+n whenever n is a nonnegative 
integer. 
Prove that 


H, +H, +- +H, = (n+ 1H, —n. 


Use mathematical induction in Exercises 31-37 to prove di- 
visibility facts. 


31. 


32. 


33. 


34. 


* 35. 


* 36. 


* 37, 


Prove that 2 divides n? + n whenever n is a positive inte- 
ger. 

Prove that 3 divides n? + 2n whenever n is a positive 
integer. 

Prove that 5 divides n° — n whenever n is a nonnegative 
integer. 

Prove that 6 divides n? — n whenever n is a nonnegative 
integer. 

Prove that n? — 1 is divisible by 8 whenever n is an odd 
positive integer. 

Prove that 21 divides 4"+! + 57"-! whenever n is a posi- 
tive integer. 


Prove that if n is a positive integer, then 133 divides 
117+! + 122-1, 


Use mathematical induction in Exercises 38—46 to prove re- 
sults about sets. 


38. 


39. 


Prove that if A,,A,,...,A, and B}, B,,...,B, are sets 
such that A; G B; for j = 1,2, ..., n, then 
Ua c Us, 
j=1 
Prove that if A,,A,,...,A, and B}, B,,...,B, are sets 
such that Aj Cc B; for j = 1,2, ...,n, then 
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40. 


41. 


42. 


43. 


44. 


45. 


* 46. 


Prove that if A}, A>, .. 


(ANANA = NAA DUB 
= (A; U B) N (A2 U B) N + N (A, UB). 


., A„ and B are sets, then 


Prove that if A,, A>, .. 


(A, UA, U = UA DANAB 
= (A; A B) U (A; AN B) U + U (A, NB). 


., A„ and B are sets, then 


Prove that if A,,A,,...,A, and B are sets, then 


(A; — B) A (A, — B) n+ N (A, — B) 
= (A, NA N NA,)-—B. 


Prove that if A,, A», .. 
U, then 


.,A, are subsets of a universal set 


U A, = [et Ay. 
k=l 


Prove that if A}, A>, .. 


(A, — B) U (A, — B) U = U (A, — B) 
= (A UA, U = UA,)— B. 


.,A, and B are sets, then 


Prove that a set with n elements has n(n — 1)/2 subsets 
containing exactly two elements whenever n is an integer 
greater than or equal to 2. 

Prove that a set with n elements has n(n — 1)(n — 2)/6 
subsets containing exactly three elements whenever n is 
an integer greater than or equal to 3. 


In Exercises 47 and 48 we consider the problem of placing 
towers along a straight road, so that every building on the road 
receives cellular service. Assume that a building receives cel- 
lular service if it is within one mile of a tower. 


47. 


*48. 


Devise a greedy algorithm that uses the minimum num- 
ber of towers possible to provide cell service to d build- 
ings located at positions x}, X2, ..., Xq from the start of the 
road. [Hint: At each step, go as far as possible along the 
road before adding a tower so as not to leave any build- 
ings without coverage. ] 

Use mathematical induction to prove that the algorithm 
you devised in Exercise 47 produces an optimal solution, 
that is, that it uses the fewest towers possible to provide 
cellular service to all buildings. 


Exercises 49-51 present incorrect proofs using mathemati- 
cal induction. You will need to identify an error in reasoning 
in each exercise. 


49. 


What is wrong with this “proof” that all horses are the 
same color? 

Let P(n) be the proposition that all the horses in a set of n 
horses are the same color. 


Basis Step: Clearly, P(1) is true. 


Inductive Step: Assume that P(k) is true, so that all 
the horses in any set of k horses are the same color. 
Consider any k+1 horses; number these as horses 
1,2,3,...,k,k +1. Now the first k of these horses all 
must have the same color, and the last k of these must 


50. 


51. 


52. 


* 53. 


54. 


* 55, 


also have the same color. Because the set of the first k 
horses and the set of the last k horses overlap, all k + 1 
must be the same color. This shows that P(k + 1) is true 
and finishes the proof by induction. 

What is wrong with this “proof”? 

“Theorem” For every positive integer n, $, 


i=1 
(n+ 5)°/2. 


i= 


Basis Step: The formula is true for n = 1. 


Inductive Step: Suppose that X;_;i=(n+ 3) /2. 
Then Ye i=(Y", +4). By the inductive 

eis (at sy /2+n+1= 
(m +n++)/2 +n +1 =m + 3n + 4/2 = 
(n+ 232/2 =[(n+1)+ 17/2, completing the inductive 
step. 


hypothesis, we have J}, 


What is wrong with this “proof”? 
“Theorem” For every positive integer n, if x and y are 
positive integers with max(x, y) = n, then x = y. 


Basis Step: Suppose that n = 1. If max(x, y) = 1 and x 
and y are positive integers, we have x = 1 and y = 1. 


Inductive Step: Let k be a positive integer. Assume that 
whenever max(x, y) = k and x and y are positive integers, 
then x = y. Now let max(x, y) = k + 1, where x and y are 
positive integers. Then max(x — 1, y — 1) = k, so by the 
inductive hypothesis, x — 1 = y — 1. It follows that x = y, 
completing the inductive step. 

Suppose that m and n are positive integers with m > n 
and f is a function from {1, 2, ..., m} to {1, 2, ..., n}. Use 
mathematical induction on the variable n to show that f 
is not one-to-one. 

Use mathematical induction to show that n people can di- 
vide a cake (where each person gets one or more separate 
pieces of the cake) so that the cake is divided fairly, that 
is, in the sense that each person thinks he or she got at 
least (1/n)th of the cake. [Hint: For the inductive step, 
take a fair division of the cake among the first k peo- 
ple, have each person divide their share into what this 
person thinks are k + 1 equal portions, and then have the 
(k + 1)st person select a portion from each of the k peo- 
ple. When showing this produces a fair division for k + 1 
people, suppose that person k + 1 thinks that person i got 
p; of the cake, where yo p=] 

Use mathematical induction to show that given a set of 
n+ l positive integers, none exceeding 2n, there is at 
least one integer in this set that divides another integer 
in the set. 

A knight on a chessboard can move one space horizon- 
tally (in either direction) and two spaces vertically (in 
either direction) or two spaces horizontally (in either 
direction) and one space vertically (in either direction). 
Suppose that we have an infinite chessboard, made up of 
all squares (m, n), where m and n are nonnegative integers 
that denote the row number and the column number of the 
square, respectively. Use mathematical induction to show 
that a knight starting at (0, 0) can visit every square using 


56. 


57. 


58. 


59. 


60. 


*61. 


* 62. 


** 63, 


64. 


65. 


* 66. 


a finite sequence of moves. [Hint: Use induction on the 
variable s = m + n.] 
Suppose that 
a 0 
s=[o al 
where a and b are real numbers. Show that 
w aq" 0 
wel w 
for every positive integer n. 
(Requires calculus) Use mathematical induction to 
prove that the derivative of f(x) = x” equals nx”! when- 
ever n is a positive integer. (For the inductive step, use 
the product rule for derivatives.) 
Suppose that A and B are square matrices with the prop- 
erty AB = BA. Show that AB” = B”A for every positive 
integer n. 
Suppose that m is a positive integer. Use mathematical 
induction to prove that if a and b are integers with a = b 
(mod m), then a‘ = b* (mod m) whenever k is a nonneg- 
ative integer. 
Use mathematical induction to show that —(p, V p V 
+ V Pa) is equivalent to =p, Aap, A + Ap, Whenever 
Pi P2, «++» Pn are propositions. 
Show that 
[P1 > Pz) A P2 > P3) A A Pn- > Prd 
> [1 AP2 A APn-1) > Pal 
is a tautology whenever p}, Py, ... 
where n > 2. 
Show that n lines separate the plane into (n? + n + 2)/2 
regions if no two of these lines are parallel and no three 
pass through a common point. 


Let dj, d5,...,a, be positive real numbers. The arith- 
metic mean of these numbers is defined by 


A=(a +a, +- +a,)/n 


,P, are propositions, 


and the geometric mean of these numbers is defined by 
G = (aja; = a„)!”". 


Use mathematical induction to prove that A > G. 

Use mathematical induction to prove Lemma 3 of 
Section 4.3, which states that if p is a prime and 
p | aja + a„, Where a; is an integer for i = 1, 2, 3, ..., n, 
then p | a; for some integer i. 

Show that if n is a positive integer, then 


1 


— =n. 
araz "Ak 


{a,,...4,}C{1,2,...,2} 
(Here the sum is over all nonempty subsets of the set of 
the n smallest positive integers.) 

Use the well-ordering property to show that the follow- 
ing form of mathematical induction is a valid method to 
prove that P(n) is true for all positive integers n. 


Basis Step: P(1) and P(2) are true. 


Inductive Step: For each positive integer k, if P(k) and 
P(k + 1) are both true, then P(k + 2) is true. 


67. 


* 68. 
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Show that if A,, A3, ...,A, are sets where n > 2, and for 
all pairs of integers i and j with 1 <i <j <n, either A; 
is a subset of A; or A is a subset of A,, then there is an 
integer i, 1 <i <n, such that A, is a subset of A for all 
integers j with 1 <j <n. 

A guest at a party is a celebrity if this person is known 
by every other guest, but knows none of them. There is at 
most one celebrity at a party, for if there were two, they 
would know each other. A particular party may have no 
celebrity. Your assignment is to find the celebrity, if one 
exists, at a party, by asking only one type of question— 
asking a guest whether they know a second guest. Every- 
one must answer your questions truthfully. That is, if Al- 
ice and Bob are two people at the party, you can ask Alice 
whether she knows Bob; she must answer correctly. Use 
mathematical induction to show that if there are n peo- 
ple at the party, then you can find the celebrity, if there 
is one, with 3(n — 1) questions. [Hint: First ask a ques- 
tion to eliminate one person as a celebrity. Then use the 
inductive hypothesis to identify a potential celebrity. Fi- 
nally, ask two more questions to determine whether that 
person is actually a celebrity. ] 


Suppose there are n people in a group, each aware of a scandal 
no one else in the group knows about. These people commu- 
nicate by telephone; when two people in the group talk, they 
share information about all scandals each knows about. For 
example, on the first call, two people share information, so 
by the end of the call, each of these people knows about two 
scandals. The gossip problem asks for G(n), the minimum 
number of telephone calls that are needed for all n people to 
learn about all the scandals. Exercises 69-71 deal with the 
gossip problem. 


69. 
70. 


ETI. 
*72. 


*73. 


Find G(1), G(2), G(3), and G(4). 
Use mathematical induction to prove that G(n) < 2n — 4 


for n > 4. [Hint: In the inductive step, have a new person 
call a particular person at the start and at the end.] 


Prove that G(n) = 2n — 4 forn > 4. 


Show that it is possible to arrange the numbers 1, 2, ...,7 
in arow so that the average of any two of these numbers 
never appears between them. [Hint: Show that it suffices 
to prove this fact when n is a power of 2. Then use math- 
ematical induction to prove the result when n is a power 


of 2.] 
Show that if J,, h, ..., Z, is a collection of open intervals 
on the real number line, n > 2, and every pair of these 
intervals has a nonempty intersection, that is, Z; N L +ø 
whenever 1 < į < n and 1 < j < n, then the intersection 
of all these sets is nonempty, that is, Z; N J N + N I, # Ø. 
(Recall that an open interval is the set of real numbers 
x with a < x < b, where a and b are real numbers with 


a<b.) 


Sometimes we cannot use mathematical induction to prove a 
result we believe to be true, but we can use mathematical in- 
duction to prove a stronger result. Because the inductive hy- 
pothesis of the stronger result provides more to work with, this 
process is called inductive loading. We use inductive loading 
in Exercise 74-76. 
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74. 


75. 


76. 


Show that we cannot use mathematical induction to prove 
that 24 1/j? < 2 for all positive integers n, but that this 
inequality is a consequence of the inequality proved by 
mathematical induction in Exercise 19. 

Suppose that we want to prove that 


VI/G+tVI<1 
j=l 


for all positive integers n. 

a) Show that if we try to prove this inequality using 
mathematical induction, the basis step works, but the 
inductive step fails. 

b) Show that mathematical induction can be used to 
prove the stronger inequality 


WUG+tV)!<1-1/M4+D! 
j=l 


for all positive integers n, implying that the weaker 


77. 


78. 


79. 


80. 


weaker inequality we originally tried to prove using 
mathematical induction. 
Let n be an even integer. Show that it is possible for n peo- 
ple to stand in a yard at mutually distinct distances so that 
when each person throws a pie at their nearest neighbor, 
everyone is hit by a pie. 


Construct a tiling using right triominoes of the 4x4 
checkerboard with the square in the upper left corner 
removed. 


Construct a tiling using right triominoes of the 8 x 8 
checkerboard with the square in the upper left corner 
removed. 


Prove or disprove that all checkerboards of these shapes 
can be completely covered using right triominoes when- 
ever n is a positive integer. 
a) 3x2” 

c) 3" x3" 


b) 6x2" 
d) 6” x 6" 


inequality is also true. *81. Show that a three-dimensional 2” x 2” x 2” checkerboard 
Süppóse that we Want to prow that with one | x 1 x 1 cube missing can be completely cov- 
E La orai p Í ered by 2 x 2 x 2 cubes with one 1 x 1 x 1 cube removed. 
n- 
oa < *82. Show that an nxn checkerboard with one square re- 
2 4 2n 3 aa os cae 
n moved can be completely covered using right triominoes 
for all positive integers n. ifn > 5, nis odd, and 3 | n. 
a) Show that if we try to prove this inequality using 83. Show that a 5 x 5 checkerboard with a corner square re- 
mathematical induction, the basis step works, but the moved can be tiled using right triominoes. 
inductive step fails. . : 
b) Show that mathematical induction can be used to *84. Find a 5 x5 checkerboard with a square removed that 
prove the stronger inequality cannot be tiled using right triominoes. Prove that such 
Ka orei i a tiling does not exist for this board. 
n- 
5 A Oy < 85. Use the principle of mathematical induction to show that 


V3nt+1 


for all integers n greater than 1, which, together witha 
verification for the case where n = 1, establishes the 


P(n)is true for n = b, b + 1,b+2, ..., where b is an inte- 
ger, if P(b) is true and the conditional statement P(k) > 
P(k + 1) is true for all integers k with k > b. 


Strong Induction and Well-Ordering 


5.2.1 Introduction 


In Section 5.1 we introduced mathematical induction and we showed how to use it to prove a 
variety of theorems. In this section we will introduce another form of mathematical induction, 
called strong induction, which can often be used when we cannot easily prove a result using 
mathematical induction. The basis step of a proof by strong induction is the same as a proof of 
the same result using mathematical induction. That is, in a strong induction proof that P(n) is 
true for all positive integers n, the basis step shows that P(1) is true. However, the inductive steps 
in these two proof methods are different. In a proof by mathematical induction, the inductive 
step shows that if the inductive hypothesis P(x) is true, then P(k + 1) is also true. In a proof 
by strong induction, the inductive step shows that if P(j) is true for all positive integers j not 
exceeding k, then P(k + 1) is true. That is, for the inductive hypothesis we assume that P(/) is 
true for j = 1, 2,...,k. 

The validity of both mathematical induction and strong induction follow from the well- 
ordering property in Appendix 1. In fact, mathematical induction, strong induction, and well- 
ordering are all equivalent principles (as shown in Exercises 41, 42, and 43). That is, the validity 
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of each can be proved from either of the other two. This means that a proof using one of these 
two principles can be rewritten as a proof using either of the other two principles. Just as it is 
sometimes the case that it is much easier to see how to prove a result using strong induction 
rather than mathematical induction, it is sometimes easier to use well-ordering than one of the 
two forms of mathematical induction. In this section we will give some examples of how the 
well-ordering property can be used to prove theorems. 


5.2.2 Strong Induction 


Before we illustrate how to use strong induction, we state this principle again. 


STRONG INDUCTION To prove that P(n) is true for all positive integers n, where P(n) is a 
propositional function, we complete two steps: 


BASIS STEP: We verify that the proposition P(1) is true. 


INDUCTIVE STEP: We show that the conditional statement [P(1) A P(2) A + A P(k)] > 
P(k + 1) is true for all positive integers k. 


Note that when we use strong induction to prove that P(n) is true for all positive integers n, 
our inductive hypothesis is the assumption that P(j) is true for j = 1, 2, ..., k. That is, the induc- 
tive hypothesis includes all k statements P(1), P(2), ..., P(k). Because we can use all k state- 
ments P(1), P(2), ..., P(k) to prove P(k + 1), rather than just the statement P(x) as in a proof by 
mathematical induction, strong induction is a more flexible proof technique. Because of this, 
some mathematicians prefer to always use strong induction instead of mathematical induction, 
even when a proof by mathematical induction is easy to find. 

You may be surprised that mathematical induction and strong induction are equivalent. 
That is, each can be shown to be a valid proof technique assuming that the other is valid. In 
particular, any proof using mathematical induction can also be considered to be a proof by 
strong induction because the inductive hypothesis of a proof by mathematical induction is part 
of the inductive hypothesis in a proof by strong induction. That is, if we can complete the induc- 
tive step of a proof using mathematical induction by showing that P(k + 1) follows from P(k) 
for every positive integer k, then it also follows that P(k + 1) follows from all the statements 
P(A), P(2), ..., P(k), because we are assuming that not only P(x) is true, but also more, namely, 
that the k — 1 statements P(1), P(2), ..., P(k — 1) are true. However, it is much more awkward to 
convert a proof by strong induction into a proof using the principle of mathematical induction. 
(See Exercise 42.) 

Strong induction is sometimes called the second principle of mathematical induction 
or complete induction. When the terminology “complete induction” is used, the principle of 
mathematical induction is called incomplete induction, a technical term that is a somewhat 
unfortunate choice because there is nothing incomplete about the principle of mathematical 
induction; after all, it is a valid proof technique. 


STRONG INDUCTION AND THE INFINITE LADDER To better understand strong induc- 
tion, consider the infinite ladder in Section 5.1. Strong induction tells us that we can reach all 
rungs if 


1. we can reach the first rung, and 
2. for every positive integer k, if we can reach all the first k rungs, then we can reach the (k + 1)st 
rung. 


That is, if P(n) is the statement that we can reach the nth rung of the ladder, by strong induction 
we know that P(n) is true for all positive integers n, because (1) tells us P(1) is true, completing 
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EXAMPLE 1 


the basis step and (2) tells us that P(1) A P(2) A «+ A P(k) implies P(k + 1), completing the in- 
ductive step. 

Example | illustrates how strong induction can help us prove a result that cannot easily be 
proved using the principle of mathematical induction. 


Suppose we can reach the first and second rungs of an infinite ladder, and we know that if we 
can reach a rung, then we can reach two rungs higher. Can we prove that we can reach every 
rung using the principle of mathematical induction? Can we prove that we can reach every rung 
using strong induction? 


Solution: We first try to prove this result using the principle of mathematical induction. 


BASIS STEP: The basis step of such a proof holds; here it simply verifies that we can reach the 
first rung. 


ATTEMPTED INDUCTIVE STEP: The inductive hypothesis is the statement that we can reach 
the kth rung of the ladder. To complete the inductive step, we need to show that if we assume 
the inductive hypothesis for the positive integer k, namely, if we assume that we can reach the 
kth rung of the ladder, then we can show that we can reach the (k + 1)st rung of the ladder. 
However, there is no obvious way to complete this inductive step because we do not know from 
the given information that we can reach the (k + 1)st rung from the kth rung. After all, we only 
know that if we can reach a rung we can reach the rung two higher. 


Now consider a proof using strong induction. 


BASIS STEP: The basis step is the same as before; it simply verifies that we can reach the first 
rung. 


INDUCTIVE STEP: The inductive hypothesis states that we can reach each of the first k rungs. 
To complete the inductive step, we need to show that if we assume that the inductive hypothesis 
is true, that is, if we can reach each of the first k rungs, then we can reach the (k + 1)st rung. 
We already know that we can reach the second rung. We can complete the inductive step by 
noting that as long as k > 2, we can reach the (k + 1)st rung from the (k — 1)st rung because 
we know we can climb two rungs from a rung we can already reach, and because k — 1 < k, by 
the inductive hypothesis we can reach the (k — 1)st rung. This completes the inductive step and 
finishes the proof by strong induction. 


We have proved that if we can reach the first two rungs of an infinite ladder and for every 
positive integer k if we can reach all the first k rungs then we can reach the (k + 1)st rung, then 
we can reach all rungs of the ladder. 


5.2.3 Examples of Proofs Using Strong Induction 


Now that we have both mathematical induction and strong induction, how do we decide which 
method to apply in a particular situation? Although there is no cut-and-dried answer, we can 
supply some useful pointers. In practice, you should use mathematical induction when it is 
straightforward to prove that P(k) => P(k + 1) is true for all positive integers k. This is the case 
for all the proofs in the examples in Section 5.1. In general, you should restrict your use of 
the principle of mathematical induction to such scenarios. Unless you can clearly see that the 
inductive step of a proof by mathematical induction goes through, you should attempt a proof 
by strong induction. That is, use strong induction and not mathematical induction when you see 
how to prove that P(k + 1) is true from the assumption that P(j) is true for all positive integers j 
not exceeding k, but you cannot see how to prove that P(k + 1) follows from just P(k). Keep this 
in mind as you examine the proofs in this section. For each of these proofs, consider why strong 
induction works better than mathematical induction. 


EXAMPLE 2 


Extra » 
Examples 


EXAMPLE 3 
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We will illustrate how strong induction is employed in Examples 2—4. In these examples, 
we will prove a diverse collection of results. Pay particular attention to the inductive step in 
each of these examples, where we show that a result P(k + 1) follows under the assumption that 
P(j) holds for all positive integers j not exceeding k, where P(n) is a propositional function. 

Before we present these examples, note that we can slightly modify strong induction to 
handle a wider variety of situations. In particular, we can adapt strong induction to handle cases 
where the inductive step is valid only for integers greater than a particular integer. Let b be a 
fixed integer and j a fixed positive integer. The form of strong induction we need tells us that 
P(n) is true for all integers n with n > b if we can complete these two steps: 


BASIS STEP: We verify that the propositions P(b), P(b + 1), ..., P(b + j) are true. 


INDUCTIVE STEP: We show that [P(b) A P(b + 1) A © A P(k)] > Pk + 1) is true for every 
integer k > b +j. 


That this alternative form is equivalent to strong induction is left as Exercise 28. 

We begin with one of the most prominent uses of strong induction, the part of the fun- 
damental theorem of arithmetic that tells us that every positive integer can be written as the 
product of primes. 


Show that if n is an integer greater than 1, then n can be written as the product of primes. 


Solution: Let P(n) be the proposition that n can be written as the product of primes. 


BASIS STEP: P(2) is true, because 2 can be written as the product of one prime, itself. (Note 
that P(2) is the first case we need to establish.) 


INDUCTIVE STEP: The inductive hypothesis is the assumption that P(j) is true for all 
integers j with 2 < j < k, that is, the assumption that j can be written as the product of primes 
whenever j is a positive integer at least 2 and not exceeding k. To complete the inductive step, 
it must be shown that P(k + 1) is true under this assumption, that is, that k + 1 is the product 
of primes. 

There are two cases to consider, namely, when k + 1 is prime and when k + 1 is composite. 
If k+ 1 is prime, we immediately see that P(k + 1) is true. Otherwise, k + 1 is composite and 
can be written as the product of two positive integers a and b with 2 < a < b < k + 1. Because 
both a and b are integers at least 2 and not exceeding k, we can use the inductive hypothesis to 
write both a and b as the product of primes. Thus, if k + 1 is composite, it can be written as the 
product of primes, namely, those primes in the factorization of a and those in the factorization 
of b. < 


Remark: Because 1 can be thought of as an empty product of primes, that is, the product of no 
primes, we could have started the proof in Example 2 with P(1) as the basis step. We chose not 
to do so because many people find this confusing. 


Example 2 completes the proof of the fundamental theorem of arithmetic, which asserts that 
every nonnegative integer can be written uniquely as the product of primes in nondecreasing 
order. We showed in Section 4.3 that an integer has at most one such factorization into primes. 
Example 2 shows there is at least one such factorization. 

Next, we show how strong induction can be used to prove that a player has a winning strategy 
in a game. 


Consider a game in which two players take turns removing any positive number of matches they 
want from one of two piles of matches. The player who removes the last match wins the game. 
Show that if the two piles contain the same number of matches initially, the second player can 
always guarantee a win. 
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EXAMPLE 4 


Solution: Let n be the number of matches in each pile. We will use strong induction to prove 
P(n), the statement that the second player can win when there are initially n matches in each 
pile. 


BASIS STEP: When n = 1, the first player has only one choice, removing one match from one 
of the piles, leaving a single pile with a single match, which the second player can remove to 
win the game. 


INDUCTIVE STEP: The inductive hypothesis is the statement that P(/) is true for all j with 
1 <j < k, that is, the assumption that the second player can always win whenever there are j 
matches, where 1 < j < k in each of the two piles at the start of the game. We need to show that 
P(k + 1) is true, that is, that the second player can win when there are initially k + 1 matches 
in each pile, under the assumption that P(j) is true for j = 1, 2, ..., k. So suppose that there are 
k + 1 matches in each of the two piles at the start of the game and suppose that the first player 
removes r matches (1 < r < k) from one of the piles, leaving k + 1 — r matches in this pile. 
By removing the same number of matches from the other pile, the second player creates the 
situation where there are two piles each with k + 1 — r matches. Because 1 <k+1-—r<k, 
we can now use the inductive hypothesis to conclude that the second player can always win. 
We complete the proof by noting that if the first player removes all k + 1 matches from one of 
the piles, the second player can win by removing all the remaining matches. < 


Using the principle of mathematical induction, instead of strong induction, to prove the 
results in Examples 2 and 3 is difficult. However, as Example 4 shows, some results can be 
readily proved using either the principle of mathematical induction or strong induction. 


Prove that every amount of postage of 12 cents or more can be formed using just 4-cent and 
5-cent stamps. 


Solution: We will prove this result using the principle of mathematical induction. Then we will 
present a proof using strong induction. Let P(n) be the statement that postage of n cents can be 
formed using 4-cent and 5-cent stamps. 

We begin by using the principle of mathematical induction. 


BASIS STEP: Postage of 12 cents can be formed using three 4-cent stamps. 


INDUCTIVE STEP: The inductive hypothesis is the statement that P(k) is true. That is, under 
this hypothesis, postage of k cents can be formed using 4-cent and 5-cent stamps. To complete 
the inductive step, we need to show that when we assume P(k) is true, then P(k + 1) is also true 
where k > 12. That is, we need to show that if we can form postage of k cents, then we can form 
postage of k + 1 cents. So, assume the inductive hypothesis is true; that is, assume that we can 
form postage of k cents using 4-cent and 5-cent stamps. We consider two cases, when at least 
one 4-cent stamp has been used and when no 4-cent stamps have been used. First, suppose that 
at least one 4-cent stamp was used to form postage of k cents. Then we can replace this stamp 
with a 5-cent stamp to form postage of k + 1 cents. But if no 4-cent stamps were used, we can 
form postage of k cents using only 5-cent stamps. Moreover, because k > 12, we needed at least 
three 5-cent stamps to form postage of k cents. So, we can replace three 5-cent stamps with four 
4-cent stamps to form postage of k + 1 cents. This completes the inductive step. 


Because we have completed the basis step and the inductive step, we know that P(n) is true 
for alln > 12. That is, we can form postage of n cents, where n > 12 using just 4-cent and 5-cent 
stamps. This completes the proof by mathematical induction. 


Next, we will use strong induction to prove the same result. In this proof, in the basis 
step we show that P(12), P(13), P(14), and P(15) are true, that is, that postage of 12, 13, 14, 
or 15 cents can be formed using just 4-cent and 5-cent stamps. In the inductive step we show 
how to get postage of k + 1 cents for k > 15 from postage of k — 3 cents. 
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BASIS STEP: We can form postage of 12, 13, 14, and 15 cents using three 4-cent stamps, two 
4-cent stamps and one 5-cent stamp, one 4-cent stamp and two 5-cent stamps, and three 5-cent 
stamps, respectively. This shows that P(12), P(13), P(14), and P(15) are true. This completes 
the basis step. 


INDUCTIVE STEP: The inductive hypothesis is the statement that P(/) is true for 12 <j < k, 
where k is an integer with k > 15. To complete the inductive step, we assume that we can form 
postage of j cents, where 12 < j < k. We need to show that under the assumption that P(k + 1) 
is true, we can also form postage of k + 1 cents. Using the inductive hypothesis, we can assume 
that P(k — 3) is true because k — 3 > 12, that is, we can form postage of k — 3 cents using just 
4-cent and 5-cent stamps. To form postage of k +1 cents, we need only add another 4-cent 
stamp to the stamps we used to form postage of k — 3 cents. That is, we have shown that if the 
inductive hypothesis is true, then P(k + 1) is also true. This completes the inductive step. 


Because we have completed the basis step and the inductive step of a strong induction proof, 
we know by strong induction that P(n) is true for all integers n with n > 12. That is, we know 
that every postage of n cents, where n is at least 12, can be formed using 4-cent and 5-cent 
stamps. This finishes the proof by strong induction. 

(There are other ways to approach this problem besides those described here. Can you find 
a solution that does not use mathematical induction?) < 


5.2.4 Using Strong Induction in Computational Geometry 


Our next example of strong induction will come from computational geometry, the part of 
discrete mathematics that studies computational problems involving geometric objects. Com- 
putational geometry is used extensively in computer graphics, computer games, robotics, scien- 
tific calculations, and a vast array of other areas. Before we can present this result, we introduce 
some terminology, possibly familiar from earlier studies in geometry. 

A polygon is a closed geometric figure consisting of a sequence of line segments 
Sp S% ..., Sp, Called sides. Each pair of consecutive sides, s; and s;,,,/= 1, 2, ..., n — 1, as well 
as the last side s,, and the first side s,, of the polygon meet at a common endpoint, called a ver- 
tex. A polygon is called simple if no two nonconsecutive sides intersect. Every simple polygon 
divides the plane into two regions: its interior, consisting of the points inside the curve, and its 
exterior, consisting of the points outside the curve. This last fact is surprisingly complicated to 
prove. It is a special case of the deceptively simple Jordan curve theorem, an important result 
with a rich history, which tells us that every simple curve divides the plane into two regions; 
see [Or00], for example. 

A polygon is called convex if every line segment connecting, two points in the interior of 
the polygon lies entirely inside the polygon. (A polygon that is not convex is said to be noncon- 
vex.) Figure 1 displays some polygons; polygons (a) and (b) are convex, but polygons (c) and 
(d) are not. A diagonal of a simple polygon is a line segment connecting two nonconsecutive 
vertices of the polygon, and a diagonal is called an interior diagonal if it lies entirely inside 
the polygon, except for its endpoints. For example, in polygon (d), the line segment connecting 
a and f is an interior diagonal, but the line segment connecting a and d is a diagonal that is not 
an interior diagonal. 


[\ Of 4 N, < < P is an interior diagonal 


ad is not an interior diagonal 


(a) 


FIGURE 1 Convex and nonconvex polygons. 
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THEOREM 1 


LEMMA 1 


Two different triangulations of a simple polygon with seven sides into five triangles, 
shown with dotted lines and with dashed lines, respectively 


FIGURE 2 Triangulations of a polygon. 


One of the most basic operations of computational geometry involves dividing a sim- 
ple polygon into triangles by adding nonintersecting diagonals. This process is called 
triangulation. Note that a simple polygon can have many different triangulations, as shown 
in Figure 2. Perhaps the most basic fact in computational geometry is that it is possible to trian- 
gulate every simple polygon, as we state in Theorem 1. Furthermore, this theorem tells us that 
every triangulation of a simple polygon with n sides includes n — 2 triangles. 


A simple polygon with n sides, where n is an integer with n > 3, can be triangulated into 
n — 2 triangles. 


It seems obvious that we should be able to triangulate a simple polygon by successively 
adding interior diagonals. Consequently, a proof by strong induction seems promising. However, 
such a proof requires this crucial lemma. 


Every simple polygon with at least four sides has an interior diagonal. 


Although Lemma | seems particularly simple, it is surprisingly tricky to prove. In fact, as 
recently as 30 years ago, a variety of incorrect proofs thought to be correct were commonly seen 
in books and articles. We defer the proof of Lemma 1 until after we prove Theorem 1. It is not 
uncommon to prove a theorem pending the later proof of an important lemma. 


Proof (of Theorem 1): We will prove this result using strong induction. Let T(n) be the state- 
ment that every simple polygon with n sides can be triangulated into n — 2 triangles. 


BASIS STEP: T(3)is true because a simple polygon with three sides is a triangle. We do not need 
to add any diagonals to triangulate a triangle; it is already triangulated into one triangle, itself. 
Consequently, every simple polygon with n = 3 has can be triangulated into n — 2 = 3 -2 = 1 
triangle. 


INDUCTIVE STEP: For the inductive hypothesis, we assume that T(j) is true for all 
integers j with 3<j<k. That is, we assume that we can triangulate a simple polygon 
with j sides into j — 2 triangles whenever 3 < j < k. To complete the inductive step, we must 
show that when we assume the inductive hypothesis, P(k + 1) is true, that is, that every simple 
polygon with k + 1 sides can be triangulated into (k + 1) — 2 =k — 1 triangles. 

So, suppose that we have a simple polygon P with k + 1 sides. Because k + 1 > 4, Lemma 1 
tells us that P has an interior diagonal ab. Now, ab splits P into two simple polygons Q, 
with s sides, and R, with ¢ sides. The sides of Q and R are the sides of P, together with the 
side ab, which is a side of both Q and R. Note that 3 < s < k and 3 < t < k because both Q 
and R have at least one fewer side than P does (after all, each of these is formed from P by 
deleting at least two sides and replacing these sides by the diagonal ab). Furthermore, the num- 
ber of sides of P is two less than the sum of the numbers of sides of Q and the number of sides 
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T is the triangle abc 
p is the vertex of P inside T such that the Z bap is smallest 


bp must be an interior diagonal of P 


FIGURE 3 Constructing an interior diagonal of a simple polygon. 


of R, because each side of P is a side of either Q or of R, but not both, and the diagonal ab is a 
side of both Q and R, but not P. That is, k+ 1 =s +t-2. 

We now use the inductive hypothesis. Because both 3 < s < k and 3 < t < k, by the induc- 
tive hypothesis we can triangulate Q and R into s — 2 and t — 2 triangles, respectively. Next, note 
that these triangulations together produce a triangulation of P. (Each diagonal added to triangu- 
late one of these smaller polygons is also a diagonal of P.) Consequently, we can triangulate P 
into a total of (s — 2) + (t — 2) = s +t — 4 = (k + 1) — 2 triangles. This completes the proof by 
strong induction. That is, we have shown that every simple polygon with n sides, where n > 3, 
can be triangulated into n — 2 triangles. < 

We now return to our proof of Lemma 1. We present a proof published by Chung-Wu Ho 
[H075]. Note that although this proof may be omitted without loss of continuity, it does provide 
a correct proof of a result proved incorrectly by many mathematicians. 


Proof: Suppose that P is a simple polygon drawn in the plane. Furthermore, suppose that b is 
the point of P or in the interior of P with the least y-coordinate among the vertices with the 
smallest x-coordinate. Then b must be a vertex of P, for if it is an interior point, there would 
have to be a vertex of P with a smaller x-coordinate. Two other vertices each share an edge 
with b, say a and c. It follows that the angle in the interior of P formed by ab and bc must be 
less than 180 degrees (otherwise, there would be points of P with smaller x-coordinates than b). 

Now let T be the triangle /\abc. If there are no vertices of P on or inside T, we can 
connect a and c to obtain an interior diagonal. On the other hand, if there are vertices of P 
inside T, we will find a vertex p of P on or inside T such that bp is an interior diagonal. (This 
is the tricky part. Ho noted that in many published proofs of this lemma a vertex p was found 
such that bp was not necessarily an interior diagonal of P. See Exercise 21.) The key is to 
select a vertex p such that the angle Zbap is smallest. To see this, note that the ray starting 
at a and passing through p hits the line segment bc at a point, say q. It then follows that the 
triangle /\bagq cannot contain any vertices of P in its interior. Hence, we can connect b and p 
to produce an interior diagonal of P. Locating this vertex p is illustrated in Figure 3. < 


5.2.5 Proofs Using the Well-Ordering Property 


The validity of both the principle of mathematical induction and strong induction follows from 
a fundamental axiom of the set of integers, the well-ordering property (see Appendix 1). The 
well-ordering property states that every nonempty set of nonnegative integers has a least ele- 
ment. We will show how the well-ordering property can be used directly in proofs. Furthermore, 
it can be shown (see Exercises 41, 42, and 43) that the well-ordering property, the principle of 
mathematical induction, and strong induction are all equivalent. That is, the validity of each of 
these three proof techniques implies the validity of the other two techniques. In Section 5.1 we 
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EXAMPLE 5 


Extra > 
Examples 


EXAMPLE 6 


Exercises 


1. Use strong induction to show that if you can run one mile 


showed that the principle of mathematical induction follows from the well-ordering property. 
The other parts of this equivalence are left as Exercises 31, 42, and 43. 


THE WELL-ORDERING PROPERTY Every nonempty set of nonnegative integers has a 
least element. 


The well-ordering property can be used directly in proofs, as Example 5 illustrates. 


Use the well-ordering property to prove the division algorithm. Recall that the division algo- 
rithm states that if a is an integer and d is a positive integer, then there are unique integers q and 
r withO <r < danda = dq +r. 


Solution: Let S be the set of nonnegative integers of the form a — dq, where q is an integer. This 
set is nonempty because —dg can be made as large as desired (taking q to be a negative integer 
with large absolute value). By the well-ordering property, S has a least element r = a — dqp. 
The integer r is nonnegative. It is also the case that r < d. If it were not, then there would 
be a smaller nonnegative element in S, namely, a — d(qọ + 1). To see this, suppose that r > d. 
Because a = dqy + r, it follows that a — d(qy + 1) = (a — dqọ) — d = r — d > 0. Consequently, 
there are integers q and r with O <r < d. The proof that q and r are unique is left as 
Exercise 37. < 


In a round-robin tournament every player plays every other player exactly once and each match 
has a winner and a loser. We say that the players p4, Po, ..., Pm form a cycle if p; beats po, pr 
beats p3, ..., Pm-1 beats p, and p,,, beats pı. Use the well-ordering property to show that if there 
is a cycle of length m (m > 3) among the players in a round-robin tournament, there must be a 
cycle of three of these players. 


Solution: We assume that there is no cycle of three players. Because there is at least one cycle 
in the round-robin tournament, the set of all positive integers n for which there is a cycle of 
length n is nonempty. By the well-ordering property, this set of positive integers has a least 
element k, which by assumption must be greater than three. Consequently, there exists a cycle 
of players pj, P2» P3, -.-, Py and no shorter cycle exists. 

Because there is no cycle of three players, we know that k > 3. Consider the first three 
elements of this cycle, p,, pọ, and p3. There are two possible outcomes of the match between 
pı and p}. If p, beats p,, it follows that p,, p2, p3 is a cycle of length three, contradicting our 
assumption that there is no cycle of three players. Consequently, it must be the case that p, 
beats p;. This means that we can omit p, from the cycle p,, p>, P3, ..., Pp, to obtain the cycle 
Pi Px Pa «++» P Of length k — 1, contradicting the assumption that the smallest cycle has length 
k. We conclude that there must be a cycle of length three. < 


a) Show that the statements P(8), P(9), and P(10) are 


or two miles, and if you can always run two more miles 
once you have run a specified number of miles, then you 
can run any number of miles. 

. Use strong induction to show that all dominoes fall in 
an infinite arrangement of dominoes if you know that the 
first three dominoes fall, and that when a domino falls, the 
domino three farther down in the arrangement also falls. 
. Let P(n) be the statement that a postage of n cents can be 
formed using just 3-cent stamps and 5-cent stamps. The 
parts of this exercise outline a strong induction proof that 
P(n) is true for all integers n > 8. 


true, completing the basis step of a proof by strong 
induction that P(n) is true for all integers n > 8. 


b) What is the inductive hypothesis of a proof by strong 
induction that P(n) is true for all integers n > 8? 


c) What do you need to prove in the inductive step of 
a proof by strong induction that P(n) is true for all 
integers n > 8? 

d) Complete the inductive step for k > 10. 

e) Explain why these steps show that P(n) is true when- 
ever n > 8. 


4. 


*9, 


10. 


11. 


Let P(n) be the statement that a postage of n cents can be 
formed using just 4-cent stamps and 7-cent stamps. The 
parts of this exercise outline a strong induction proof that 
P(n) is true for all integers n > 18. 


a) Show that the statements P(18), P(19), P(20), and 
P(21) are true, completing the basis step of a proof by 
strong induction that P(n) is true for all integers n > 18. 

b) What is the inductive hypothesis of a proof by strong 
induction that P(n) is true for all integers n > 18? 

c) What do you need to prove in the inductive step of a 
proof that P(n) is true for all integers n > 18? 

d) Complete the inductive step for k > 21. 

e) Explain why these steps show that P(n) is true for all 
integers n > 18. 


. a) Determine which amounts of postage can be formed 


using just 4-cent and 1 1-cent stamps. 

b) Prove your answer to (a) using the principle of math- 
ematical induction. Be sure to state explicitly your 
inductive hypothesis in the inductive step. 

c) Prove your answer to (a) using strong induction. How 
does the inductive hypothesis in this proof differ from 
that in the inductive hypothesis for a proof using 
mathematical induction? 


. a) Determine which amounts of postage can be formed 


using just 3-cent and 10-cent stamps. 

b) Prove your answer to (a) using the principle of math- 
ematical induction. Be sure to state explicitly your 
inductive hypothesis in the inductive step. 

c) Prove your answer to (a) using strong induction. How 
does the inductive hypothesis in this proof differ from 
that in the inductive hypothesis for a proof using 
mathematical induction? 


. Which amounts of money can be formed using just two- 


dollar bills and five-dollar bills? Prove your answer using 
strong induction. 


. Suppose that a store offers gift certificates in denomina- 


tions of 25 dollars and 40 dollars. Determine the possible 
total amounts you can form using these gift certificates. 
Prove your answer using strong induction. 


Use strong induction to prove that V2 is irrational. [Hint: 


Let P(n) be the statement that V2 + n/b for any positive 
integer b.] 

Assume that a chocolate bar consists of n squares ar- 
ranged in a rectangular pattern. The entire bar, or any 
smaller rectangular piece of the bar, can be broken along 
a vertical or a horizontal line separating the squares. 
Assuming that only one piece can be broken at a time, 
determine how many breaks you must successively make 
to break the bar into n separate squares. Use strong in- 
duction to prove your answer. 

Consider this variation of the game of Nim. The game 
begins with n matches. Two players take turns removing 
matches, one, two, or three at a time. The player removing 
the last match loses. Use strong induction to show that 
if each player plays the best strategy possible, the first 
player wins if n = 4j, 4j + 2, or 4j + 3 for some nonnega- 
tive integer j and the second player wins in the remaining 
case when n = 4j + 1 for some nonnegative integer j. 


12. 


13; 


14. 


15. 


* 16. 


17. 


* 18, 


*19, 
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Use strong induction to show that every positive integer 
can be written as a sum of distinct powers of two, that is, 
as a sum of a subset of the integers 29=1,2!=2,2?=4, 
and so on. [Hint: For the inductive step, separately con- 
sider the case where k + 1 is even and where it is odd. 
When it is even, note that (k + 1)/2 is an integer. ] 


A jigsaw puzzle is put together by successively joining 
pieces that fit together into blocks. A move is made each 
time a piece is added to a block, or when two blocks 
are joined. Use strong induction to prove that no matter 
how the moves are carried out, exactly n — 1 moves are 
required to assemble a puzzle with n pieces. 


Suppose you begin with a pile of n stones and split this 
pile into n piles of one stone each by successively split- 
ting a pile of stones into two smaller piles. Each time you 
split a pile you multiply the number of stones in each 
of the two smaller piles you form, so that if these piles 
have r and s stones in them, respectively, you compute 
rs. Show that no matter how you split the piles, the sum 
of the products computed at each step equals n(n — 1)/2. 


Prove that the first player has a winning strategy for the 
game of Chomp, introduced in Example 12 in Section 
1.8, if the initial board is square. [Hint: Use strong in- 
duction to show that this strategy works. For the first 
move, the first player chomps all cookies except those in 
the left and top edges. On subsequent moves, after the 
second player has chomped cookies on either the top or 
left edge, the first player chomps cookies in the same 
relative positions in the left or top edge, respectively. ] 


Prove that the first player has a winning strategy for the 
game of Chomp, introduced in Example 12 in Section 
1.8, if the initial board is two squares wide, that is, a 
2 x n board. [Hint: Use strong induction. The first move 
of the first player should be to chomp the cookie in the 
bottom row at the far right.] 


Use strong induction to show that if a simple polygon 
with at least four sides is triangulated, then at least two 
of the triangles in the triangulation have two sides that 
border the exterior of the polygon. 


Use strong induction to show that when a simple poly- 


gon P with consecutive vertices v}, vy, ..., Vv, is trian- 
gulated into n — 2 triangles, the n — 2 triangles can be 
numbered 1, 2, ..., n — 2 so that v, is a vertex of triangle 


ifori=1,2,...,n—2. 


Pick’s theorem says that the area of a simple polygon P 
in the plane with vertices that are all lattice points (that is, 
points with integer coordinates) equals /(P) +B(P)/2 — 1, 
where /(P) and B(P) are the number of lattice points in 
the interior of P and on the boundary of P, respectively. 
Use strong induction on the number of vertices of P to 
prove Pick’s theorem. [Hint: For the basis step, first prove 
the theorem for rectangles, then for right triangles, and 
finally for all triangles by noting that the area of a tri- 
angle is the area of a larger rectangle containing it with 
the areas of at most three triangles subtracted. For the 
inductive step, take advantage of Lemma 1.] 
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**20. Suppose that P is a simple polygon with vertices 


21. 


Vi Vz, -.-,V, listed so that consecutive vertices are con- 
nected by an edge, and v, and v, are connected by an 
edge. A vertex v; is called an ear if the line segment 
connecting the two vertices adjacent to v, is an interior 
diagonal of the simple polygon. Two ears v; and v, are 
called nonoverlapping if the interiors of the triangles 
with vertices v, and its two adjacent vertices and v; and 
its two adjacent vertices do not intersect. Prove that every 
simple polygon with at least four vertices has at least two 
nonoverlapping ears. 

In the proof of Lemma 1 we mentioned that many in- 
correct methods for finding a vertex p such that the 
line segment bp is an interior diagonal of P have been 
published. This exercise presents some of the incorrect 
ways p has been chosen in these proofs. Show, by con- 
sidering one of the polygons drawn here, that for each of 
these choices of p, the line segment bp is not necessarily 
an interior diagonal of P. 


a) pis the vertex of P such that the angle Zabp is small- 
est. 

b) pis the vertex of P with the least x-coordinate (other 
than b). 

c) pis the vertex of P that is closest to b. 


e 


Exercises 22 and 23 present examples that show inductive 
loading can be used to prove results in computational geom- 


etry. 
*22. 


23. 


Let P(n) be the statement that when nonintersecting di- 
agonals are drawn inside a convex polygon with n sides, 
at least two vertices of the polygon are not endpoints of 
any of these diagonals. 


a) Show that when we attempt to prove P(n) for all 
integers n with n > 3 using strong induction, the in- 
ductive step does not go through. 

b) Show that we can prove that P(n) is true for all inte- 
gers n with n > 3 by proving by strong induction the 
stronger assertion Q(n), for n > 4, where Q(n) states 
that whenever nonintersecting diagonals are drawn 
inside a convex polygon with n sides, at least two 
nonadjacent vertices are not endpoints of any of these 
diagonals. 

Let E(n) be the statement that in a triangulation of a sim- 

ple polygon with n sides, at least one of the triangles in 

the triangulation has two sides bordering the exterior of 
the polygon. 


*24. 


25. 


26. 


27. 


28. 


29. 


a) Explain where a proof using strong induction that 
E(n) is true for all integers n > 4 runs into difficulties. 


b) Show that we can prove that E(n) is true for all inte- 
gers n > 4 by proving by strong induction the stronger 
statement T(n) for all integers n > 4, which states 
that in every triangulation of a simple polygon, at 
least two of the triangles in the triangulation have two 
sides bordering the exterior of the polygon. 

A stable assignment, defined in the preamble to Exer- 

cise 64 in Section 3.1, is called optimal for suitors if no 

stable assignment exists in which a suitor is paired with 

a suitee whom this suitor prefers to the person to whom 

this suitor is paired in this stable assignment. Use strong 

induction to show that the deferred acceptance algorithm 
produces a stable assignment that is optimal for suitors. 


Suppose that P(n) is a propositional function. Determine 

for which positive integers n the statement P(n) must be 

true, and justify your answer, if 

a) P(1) is true; for all positive integers n, if P(n) is true, 
then P(n + 2) is true. 


b) P(1) and P(2) are true; for all positive integers n, if 
P(n) and P(n + 1) are true, then P(n + 2) is true. 


c) P(1) is true; for all positive integers n, if P(n) is true, 
then P(2n) is true. 


d) P(1) is true; for all positive integers n, if P(n) is true, 
then P(n + 1) is true. 

Suppose that P(n) is a propositional function. Determine 

for which nonnegative integers n the statement P(n) must 

be true if 

a) P(O) is true; for all nonnegative integers n, if P(n) is 
true, then P(n + 2) is true. 


b) P(O) is true; for all nonnegative integers n, if P(n) is 
true, then P(n + 3) is true. 


c) P(O) and P(1) are true; for all nonnegative integers n, 
if P(n) and P(n + 1) are true, then P(n + 2) is true. 


d) P(O) is true; for all nonnegative integers n, if P(n) is 
true, then P(n + 2) and P(n + 3) are true. 

Show that if the statement P(n) is true for infinitely many 

positive integers n and P(n + 1) > P(n) is true for all pos- 

itive integers n, then P(n) is true for all positive integers n. 


Let b be a fixed integer and j a fixed positive inte- 
ger. Show that if P(b), P(b + 1), ..., P(b +j) are true 
and [P(b) A P(b+1) A+ A P(k)] ~ P(K+1) is true 
for every integer k > b +j, then P(n) is true for all 
integers n with n > b. 


What is wrong with this “proof” by strong induction? 
“Theorem” For every nonnegative integer n, 5n = 0. 
Basis Step: 5-0 =0. 


Inductive Step: Suppose that 5j = 0 for all nonnegative 
integers j withO <j < k. Write k + 1 = i + j, where i and 
j are natural numbers less than k + 1. By the inductive 
hypothesis, 5(k + 1) = 5(i +j) = 5i+5j=0+0=0. 


* 30. 


*31. 


32. 


33. 


34. 


* 35, 


* 36. 


Find the flaw with the following “proof” that a” = 1 for 
all nonnegative integers n, whenever a is a nonzero real 
number. 


Basis Step: a? = 1 is true by the definition of a°. 


Inductive Step: Assume that af = 1 for all nonnegative 
integers j with j < k. Then note that 


kt! a-a 1-1 7 
ak-! 1 


Show that strong induction is a valid method of proof by 
showing that it follows from the well-ordering property. 


Find the flaw with the following “proof” that every 
postage of three cents or more can be formed using just 
3-cent and 4-cent stamps. 


Basis Step: We can form postage of three cents with 
a single 3-cent stamp and we can form postage of four 
cents using a single 4-cent stamp. 


Inductive Step: Assume that we can form postage of j 
cents for all nonnegative integers j with j < k using just 
3-cent and 4-cent stamps. We can then form postage of 
k + 1 cents by replacing one 3-cent stamp with a 4-cent 
stamp or by replacing two 4-cent stamps by three 3-cent 
stamps. 


Show that we can prove that P(n, k) is true for all pairs 

of positive integers n and k if we show 

a) P(1, 1) is true and P(n, k) > [P(n + 1, k) A P(n, k + 
1)] is true for all positive integers n and k. 


b) Pd, k) is true for all positive integers k, and P(n, k) > 
P(n + 1, k) is true for all positive integers n and k. 


c) P(n, 1)is true for all positive integers n, and P(n, k) > 
P(n, k + 1) is true for all positive integers n and k. 
Prove that J+ IMJ +2- (j+k-1)=n(n+1) 
(n +2)» (n+ k)/(k + 1) for all positive integers k and 

n. [Hint: Use a technique from Exercise 33.] 


Show that if a,,d5,...,a, are n distinct real numbers, 
exactly n — 1 multiplications are used to compute the 
product of these n numbers no matter how parentheses 
are inserted into their product. [Hint: Use strong induc- 
tion and consider the last multiplication. ] 


The well-ordering property can be used to show that 
there is a unique greatest common divisor of two posi- 
tive integers. Let a and b be positive integers, and let S be 


37. 


38. 


R439, 


40. 


* Al. 


* 42. 


* 43. 
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the set of positive integers of the form as + bt, where s 

and ¢ are integers. 

a) Show that S is nonempty. 

b) Use the well-ordering property to show that S has a 
smallest element c. 

c) Show that if d is a common divisor of a and b, then 
d is a divisor of c. 

d) Show that c | a and c | b. [Hint: First, assume that 
c { a. Then a= qc +r, where 0 < r < c. Show that 
r € S, contradicting the choice of c.] 

e) Conclude from (c) and (d) that the greatest common 
divisor of a and b exists. Finish the proof by showing 
that this greatest common divisor is unique. 

Let a be an integer and d be a positive integer. Show 

that the integers q and r with a = dq+rand0<r<d, 

which were shown to exist in Example 5, are unique. 

Use mathematical induction to show that a rectangu- 

lar checkerboard with an even number of cells and two 

squares missing, one white and one black, can be covered 
by dominoes. 

Can you use the well-ordering property to prove the state- 

ment: “Every positive integer can be described using no 

more than fifteen English words”? Assume the words 
come from a particular dictionary of English. [Hint: 

Suppose that there are positive integers that cannot be 

described using no more than fifteen English words. By 

well ordering, the smallest positive integer that cannot 
be described using no more than fifteen English words 
would then exist. ] 

Use the well-ordering property to show that if x and y 

are real numbers with x < y, then there is a rational 

number r with x <r < y. [Hint: Use the Archimedean 
property, given in Appendix 1, to find a positive 
integer A with A > 1/(y—~x). Then show that there is 

a rational number r with denominator A between x and y 

by looking at the numbers |x] + j/A, where j is a positive 

integer. ] 

Show that the well-ordering property can be proved 

when the principle of mathematical induction is taken as 

an axiom. 

Show that the principle of mathematical induction and 

strong induction are equivalent; that is, each can be 

shown to be valid from the other. 

Show that we can prove the well-ordering property when 

we take strong induction as an axiom instead of taking 

the well-ordering property as an axiom. 


Recursive Definitions and Structural Induction 


5.3.1 Introduction 


Sometimes it is difficult to define an object explicitly. However, it may be easy to define this 
object in terms of itself. This process is called recursion. For instance, the picture shown in Fig- 
ure 1 is produced recursively. First, an original picture is given. Then a process of successively 
superimposing centered smaller pictures on top of the previous pictures is carried out. 
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Assessment » 


FIGURE 1 A recursively defined picture. 


We can use recursion to define sequences, functions, and sets. In Section 2.4, and in most 
beginning mathematics courses, the terms of a sequence are specified using an explicit formula. 
For instance, the sequence of powers of 2 is given by a, = 2” forn = 0, 1, 2, ... . Recall from Sec- 
tion 2.4 that we can also define a sequence recursively by specifying how terms of the sequence 
are found from previous terms. The sequence of powers of 2 can also be defined by giving the 
first term of the sequence, namely, ay = 1, anda rule for finding a term of the sequence from the 
previous one, namely, a,,, = 2a, for n = 0, 1, 2, .... When we define a sequence recursively by 
specifying how terms of the sequence are found from previous terms, we can use induction to 
prove results about the sequence. 

When we define a set recursively, we specify some initial elements in a basis step and 
provide a rule for constructing new elements from those we already have in the recur- 
sive step. To prove results about recursively defined sets we use a method called structural 
induction. 


5.3.2 Recursively Defined Functions 


We use two steps to define a function with the set of nonnegative integers as its domain: 
BASIS STEP: Specify the value of the function at zero. 


RECURSIVE STEP: Give a rule for finding its value at an integer from its values at smaller 
integers. 


Such a definition is called a recursive or inductive definition. Note that a function f(n) from 
the set of nonnegative integers to the set of a real numbers is the same as a sequence ap, a), ..., 
where a; is a real number for every nonnegative integer i. So, defining a real-valued sequence 
do, 4, ... using a recurrence relation, as was done in Section 2.4, is the same as defining a 
function from the set of nonnegative integers to the set of real numbers. 


EXAMPLE 1 
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EXAMPLE 2 


EXAMPLE 3 
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Suppose that f is defined recursively by 


f(O) = 3, 
f(n +1) = 2f(n) +3. 


Find f(1), f2), fG), and f(4). 


Solution: From the recursive definition it follows that 


f(1) = 2f00) +3 =2-34+3=9, 

f(2) = 2f0) +3 =2-94+3=21, 

fG) = 2f(2)+3 =2-2143 =45, 

f(4) = 2f3) +3 =2.45 +3 = 93. F 


Recursively defined functions are well defined. That is, for every positive integer, the 
value of the function at this integer is determined in an unambiguous way. This means 
that given any positive integer, we can use the two parts of the definition to find the value 
of the function at that integer, and that we obtain the same value no matter how we ap- 
ply the two parts of the definition. This is a consequence of the principle of mathemati- 
cal induction. (See Exercise 58.) Additional examples of recursive definitions are given in 
Examples 2 and 3. 


Give a recursive definition of a”, where a is a nonzero real number and n is a nonnegative integer. 


Solution: The recursive definition contains two parts. First a° is specified, namely, a? = 1. Then 
the rule for finding a”*! from a”, namely, a"! = a - a”, for n = 0, 1, 2, 3, ..., is given. These 
two equations uniquely define a” for all nonnegative integers n. < 


Give a recursive definition of 


n 


>, ap. 


k=0 


Solution: The first part of the recursive definition is 


0 
py dk = do. 
k=0 


The second part is 


n+1 n 
> ay = 5 ak + Anti: 


k=0 k=0 
< 


In some recursive definitions of functions, the values of the function at the first k positive 
integers are specified, and a rule is given for determining the value of the function at larger inte- 
gers from its values at some or all of the preceding k integers. That recursive definitions defined 
in this way produce well-defined functions follows from strong induction (see Exercise 59). 
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EXAMPLE 4 
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THEOREM 1 


Recall from Section 2.4 that the Fibonacci numbers, fo, fi 5, ..., are defined by the equa- 
tions fy = 0, fi = 1, and 


Ía = fai Efa 


for n = 2, 3, 4, .... [We can think of the Fibonacci number f, either as the nth term of the se- 
quence of Fibonacci numbers fọ, f,, ... or as the value at the integer n of a function f(n).] 

We can use the recursive definition of the Fibonacci numbers to prove many properties of 
these numbers. We give one such property in Example 4. 


Show that whenever n > 3, f, > @~*, where a = (1 + /5)/2. 
Solution: We can use strong induction to prove this inequality. Let P(n) be the statement 
f, > a"~*, We want to show that P(n) is true whenever n is an integer greater than or equal to 3. 


BASIS STEP: First, note that 


a<2=f, o =(3+ V5)/2<3 = fp 


so P(3) and P(4) are true. 


INDUCTIVE STEP: Assume that P(j) is true, namely, that f > a/~*, for all integers j with 


3 <j < k, where k > 4. We must show that P(k + 1) is true, that is, that f.,, > a*-!. Because 
a is a solution of x? — x — 1 = 0 (as the quadratic formula shows), it follows that a =atl. 
Therefore, 


al = a gh = (a + Da = a a 41-0 H a? 4+ 0. 
By the inductive hypothesis, because k > 4, we have 
fhasd, ioe, 
Therefore, it follows that 
fet = Sit Suan > a? + ok? = a 
Hence, P(k + 1) is true. This completes the proof. < 


Remark: The inductive step of the proof by strong induction in Example 4 shows that whenever 
k > 4, P(k + 1) follows from the assumption that P( j) is true for 3 < j < k. Hence, the inductive 
step does not show that P(3) > P(4). Therefore, we had to show that P(4) is true separately. 


We can now show that the Euclidean algorithm, introduced in Section 4.3, uses O(log b) 
divisions to find the greatest common divisor of the positive integers a and b, where a > b. 


LAME’S THEOREM Leta and b be positive integers with a > b. Then the number of 
divisions used by the Euclidean algorithm to find gcd(a, b) is less than or equal to five times 
the number of decimal digits in b. 
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Proof: Recall that when the Euclidean algorithm is applied to find gcd(a, b) with a > b, this 
sequence of equations (where a = rọ and b = r,) is obtained. 


ry =rq +n OS <r, 

ri =n +r OST <h, 
Fa-2 F Tn-19n-1 + Fa 0 = Fa < Tho 
Pn-1 = nin: 


Here n divisions have been used to find r, = gcd(a, b). Note that the quotients q1, q2, .-., 4n-1 
are all at least 1. Moreover, g,, > 2, because r, < r,_,. This implies that 


Fa 2 1 = fr, 
Tn-1 2 2r, 2 2f =f, 
Tn-2 2 Tn-1 + Fn >f +h = fy, 


923+ È fa- Ffa- = fw 


b=r >n+r > fa th = n+l’ 


It follows that if n divisions are used by the Euclidean algorithm to find gcd(a, b) with a > b, 


then b>f,,,. By Example 4 we know that f,,; > a"! for n > 2, where a = (1 + V5)/2. 
Therefore, it follows that b > a@’!. Furthermore, because logy a ~ 0.208 > 1/5, we see 
that 


logjo b > (n — 1) logoa > (n — 1)/5. 


Hence, n — 1 < 5 - logo b. Now suppose that b has k decimal digits. Then b < 10* and logy) b < 
k. It follows that n — 1 < 5k, and because k is an integer, it follows that n < 5k. This finishes the 
proof. < 


Because the number of decimal digits in b, which equals |log;o b] + 1, is less than or equal 
to logio b + 1, Theorem 1 tells us that the number of divisions required to find gcd(a, b) with 


FIBONACCI (1170-1250) Fibonacci (short for filius Bonacci, or “son of Bonacci”) was also known as 
Leonardo of Pisa. He was born in the Italian commercial center of Pisa. Fibonacci was a merchant who traveled 
extensively throughout the Mideast, where he came into contact with Arabian mathematics. In his book Liber 
Abaci, Fibonacci introduced the European world to Arabic notation for numerals and algorithms for arithmetic. 
It was in this book that his well known rabbit problem (described in Section 8.1) appeared. Fibonacci also wrote 
books on geometry and trigonometry and on Diophantine equations, which involve finding integer solutions to 
equations. 
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a > b is less than or equal to 5(log,) b + 1). Because 5(log;) b + 1) is O(log b), we see that 
O(log b) divisions are used by the Euclidean algorithm to find gcd(a, b) whenever a > b. 


5.3.3 Recursively Defined Sets and Structures 


We have explored how functions can be defined recursively. We now turn our attention to how 
sets can be defined recursively. Just as in the recursive definition of functions, recursive def- 
initions of sets have two parts, a basis step and a recursive step. In the basis step, an initial 
collection of elements is specified. In the recursive step, rules for forming new elements in the 
set from those already known to be in the set are provided. Recursive definitions may also in- 
clude an exclusion rule, which specifies that a recursively defined set contains nothing other 
than those elements specified in the basis step or generated by applications of the recursive step. 
In our discussions, we will always tacitly assume that the exclusion rule holds and no element 
belongs to a recursively defined set unless it is in the initial collection specified in the basis step 
or can be generated using the recursive step one or more times. Later we will see how we can 
use a technique known as structural induction to prove results about recursively defined sets. 

Examples 5, 6, 8, and 9 illustrate the recursive definition of sets. In each example, we show 
those elements generated by the first few applications of the recursive step. 


Consider the subset S of the set of integers recursively defined by 
BASIS STEP: 3 € S. 


RECURSIVE STEP: Ifx € Sandy E S, then x+y E S. 


The new elements found to be in S are 3 by the basis step, 3 + 3 = 6 at the first application of 
the recursive step, 3 + 6 = 6 + 3 = 9 and 6 + 6 = 12 at the second application of the recursive 
step, and so on. We will show in Example 10 that S is the set of all positive multiples of 3. <4 


Recursive definitions play an important role in the study of strings. (See Chapter 13 for 
an introduction to the theory of formal languages, for example.) Recall from Section 2.4 that a 
string over an alphabet È is a finite sequence of symbols from £. We can define &*, the set of 
strings over È, recursively, as Definition 1 shows. 


The set &* of strings over the alphabet È is defined recursively by 
BASIS STEP: A € X* (where 4 is the empty string containing no symbols). 
RECURSIVE STEP: If w € &* and x E X, then wx € &*. 


GABRIEL LAME (1795-1870) Gabriel Lamé entered the Ecole Polytechnique in 1813, graduating in 1817. 
He continued his education at the Ecole des Mines, graduating in 1820. 

In 1820 Lamé went to Russia, where he was appointed director of the Schools of Highways and Trans- 
portation in St. Petersburg. Not only did he teach, but he also planned roads and bridges while in Russia. He 
returned to Paris in 1832, where he helped found an engineering firm. However, he soon left the firm, accepting 
the chair of physics at the Ecole Polytechnique, which he held until 1844. While holding this position, he was 
active outside academia as an engineering consultant, serving as chief engineer of mines and participating in 
the building of railways. 

Lamé contributed original work to number theory, applied mathematics, and thermodynamics. His best- 
known work involves the introduction of curvilinear coordinates. His work on number theory includes proving 
Fermat’s last theorem for n = 7, as well as providing the upper bound for the number of divisions used by the 
Euclidean algorithm given in this text. 


In the opinion of Gauss, one of the most important mathematicians of all time, Lamé was the foremost French mathematician of 
his time. However, French mathematicians considered him too practical, whereas French scientists considered him too theoretical. 
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The basis step of the recursive definition of strings says that the empty string belongs to &*. 
The recursive step states that new strings are produced by adding a symbol from È to the end of 
strings in &*. At each application of the recursive step, strings containing one additional symbol 
are generated. 


If È = {0, 1}, the strings found to be in &”*, the set of all bit strings, are A, specified to be in X* 
in the basis step, O and 1 formed during the first application of the recursive step, 00, 01, 10, 
and 11 formed during the second application of the recursive step, and so on. < 


Recursive definitions can be used to define operations or functions on the elements of re- 
cursively defined sets. This is illustrated in Definition 2 of the concatenation of two strings and 
Example 7 concerning the length of a string. 


Two strings can be combined via the operation of concatenation. Let X be a set of symbols 
and &* the set of strings formed from symbols in £. We can define the concatenation of two 
strings, denoted by -, recursively as follows. 


BASIS STEP: Tfw € &*, then w- A = w, where / is the empty string. 
RECURSIVE STEP: If w, E€ &* and w, E X* and x E È, then w; - (w2x) = (w; - w3)x. 


The concatenation of the strings w, and w, is often written as ww, rather than w, - w3. By 
repeated application of the recursive definition, it follows that the concatenation of two strings 
w; and w, consists of the symbols in w, followed by the symbols in w,. For instance, the con- 
catenation of w, = abra and w, = cadabra is w,;w, = abracadabra. 


Length of a String Give a recursive definition of /(w), the length of the string w. 


Solution: The length of a string can be recursively defined by 


[(A) = 0; 


(wx) = Iw) + 1 if w € d* and x E È. à 


Another important use of recursive definitions is to define well-formed formulae of various 
types. This is illustrated in Examples 8 and 9. 


Well-Formed Formulae in Propositional Logic We can define the set of well-formed for- 
mulae in propositional logic involving T, F, propositional variables, and operators from the set 
{A,A,V, >, oO}. 


BASIS STEP: T, F, and s, where s is a propositional variable, are well-formed formulae. 


RECURSIVE STEP: If E and F are well-formed formulae, then (~E), (E A F), (E V F), (E > F), 
and (E « F) are well-formed formulae. 


For example, by the basis step we know that T, F, p, and q are well-formed formulae, 
where p and q are propositional variables. From an initial application of the recursive 
step, we know that (p V q), (p > F), (F > q), and (q AF) are well-formed formulae. A 
second application of the recursive step shows that ((p V q) > (q A F)), (q V (p V q)), and 
((p > F) > T) are well-formed formulae. We leave it to the reader to show that p> A q, pq^, 
and ~ A pq are not well-formed formulae, by showing that none can be obtained using the basis 
step and one or more applications of the recursive step. < 
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Definition 3 


Basis step 


formulae consisting of variables, numerals, and operators from the set {+, —, *, /, 1} (where * 
denotes multiplication and fî denotes exponentiation) recursively. 


BASIS STEP: xis a well-formed formula if x is a numeral or a variable. 


RECURSIVE STEP: If F and G are well-formed formulae, then (F + G), (F — G), (F * G), 
(F/G), and (F fî G) are well-formed formulae. 


For example, by the basis step we see that x, y, 0, and 3 are well-formed formulae (as 
is any variable or numeral). Well-formed formulae generated by applying the recursive step 
once include (x + 3), (3 + y), (x — y), (3 — 0), (x * 3), (3 * y), 3/0), (x/y), G Tx), and (013). 
Applying the recursive step twice shows that formulae such as ((x + 3) + 3) and (x — (3 * y)) 
are well-formed formulae. [Note that (3/0) is a well-formed formula because we are concerned 
only with syntax matters here.] We leave it to the reader to show that each of the formulae x3 +, 
y * +x, and * x/y is not a well-formed formula by showing that none of them can be obtained 
from the basis step and one or more applications of the recursive step. < 


We will study trees extensively in Chapter 11. A tree is a special type of a graph; a graph 
is made up of vertices and edges connecting some pairs of vertices. We will study graphs in 
Chapter 10. We will briefly introduce them here to illustrate how they can be defined recursively. 


The set of rooted trees, where a rooted tree consists of a set of vertices containing a distin- 
guished vertex called the root, and edges connecting these vertices, can be defined recursively 
by these steps: 


BASIS STEP: A single vertex r is a rooted tree. 
RECURSIVE STEP: Suppose that T,,7,,...,7,, are disjoint rooted trees with roots 


, n 


Fi, Tp...» Fp, respectively. Then the graph formed by starting with a root r, which is not 
in any of the rooted trees T}, T», ..., T,,, and adding an edge from r to each of the vertices 
11, lp, ---»T,» iS also a rooted tree. 


In Figure 2 we illustrate some of the rooted trees formed starting with the basis step and applying 
the recursive step one time and two times. Note that infinitely many rooted trees are formed at 
each application of the recursive definition. 


Step 1 


1A AN AS 


AAA A 


FIGURE 2 Building up rooted trees. 
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FIGURE 3 Building up extended binary trees. 


Definition 4 


Binary trees are a special type of rooted trees. We will provide recursive definitions of two 
types of binary trees—full binary trees and extended binary trees. In the recursive step of the 
definition of each type of binary tree, two binary trees are combined to form a new tree with 
one of these trees designated the left subtree and the other the right subtree. In extended bi- 
nary trees, the left subtree or the right subtree can be empty, but in full binary trees this is not 
possible. Binary trees are one of the most important types of structures in computer science. 
In Chapter 11 we will see how they can be used in searching and sorting algorithms, in algo- 
rithms for compressing data, and in many other applications. We first define extended binary 
trees. 


The set of extended binary trees can be defined recursively by these steps: 
BASIS STEP: The empty set is an extended binary tree. 


RECURSIVE STEP: If T, and T, are disjoint extended binary trees, there is an extended 
binary tree, denoted by T, - T), consisting of a root r together with edges connecting the 
root to each of the roots of the left subtree 7, and the right subtree T, when these trees are 
nonempty. 


Figure 3 shows how extended binary trees are built up by applying the recursive step from one 
to three times. 

We now show how to define the set of full binary trees. Note that the difference between 
this recursive definition and that of extended binary trees lies entirely in the basis step. 
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FIGURE 4 Building up full binary trees. 
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The set of full binary trees can be defined recursively by these steps: 
BASIS STEP: There is a full binary tree consisting only of a single vertex r. 


RECURSIVE STEP: If T, and T, are disjoint full binary trees, there is a full binary tree, 
denoted by T; - T,, consisting of a root r together with edges connecting the root to each of 
the roots of the left subtree 7, and the right subtree T. 


Figure 4 shows how full binary trees are built up by applying the recursive step one and two 
times. 


5.3.4 Structural Induction 


To prove results about recursively defined sets, we generally use some form of mathematical 
induction. Example 10 illustrates the connection between recursively defined sets and mathe- 
matical induction. 


Show that the set S defined in Example 5 by specifying that 3 € S and that if x € Sandy € S, 
then x + y € S, is the set of all positive integers that are multiples of 3. 


Solution: Let A be the set of all positive integers divisible by 3. To prove that A = S, we must 
show that A is a subset of S and that S is a subset of A. To prove that A is a subset of S, we must 
show that every positive integer divisible by 3 is in S. We will use mathematical induction to 
prove this. 

Let P(n) be the statement that 3n belongs to S. The basis step holds because by the first part 
of the recursive definition of S, 3 - 1 = 3 is in S. To establish the inductive step, assume that 
P(k) is true, namely, that 3k is in S. Because 3k is in S and because 3 is in S, it follows from the 
second part of the recursive definition of S that 3k + 3 = 3(k + 1) is also in S. 

To prove that S is a subset of A, we use the recursive definition of S. First, the basis step 
of the definition specifies that 3 is in S. Because 3 = 3 - 1, all elements specified to be in S$ in 
this step are divisible by 3 and are therefore in A. To finish the proof, we must show that all 
integers in S generated using the second part of the recursive definition are in A. This consists 
of showing that x + y is in A whenever x and y are elements of S also assumed to be in A. Now 
if x and y are both in A, it follows that 3 | x and 3 | y. By part (i) of Theorem 1 of Section 4.1, 
it follows that 3 | (x + y) completing the proof. < 


EXAMPLE 11 
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In Example 10 we used mathematical induction over the set of positive integers and a re- 
cursive definition to prove a result about a recursively defined set. However, instead of using 
mathematical induction directly to prove results about recursively defined sets, we can use a 
more convenient form of induction known as structural induction. A proof by structural in- 
duction consists of two parts. These parts are 


BASIS STEP: Show that the result holds for all elements specified in the basis step of the 
recursive definition to be in the set. 


RECURSIVE STEP: Show that if the statement is true for each of the elements used to construct 
new elements in the recursive step of the definition, the result holds for these new elements. 


The validity of structural induction follows from the principle of mathematical induction 
for the nonnegative integers. To see this, let P(n) state that the claim is true for all elements 
of the set that are generated by n or fewer applications of the rules in the recursive step of 
a recursive definition. We will have established that the principle of mathematical induction 
implies the principle of structural induction if we can show that P(n) is true whenever n is a 
positive integer. In the basis step of a proof by structural induction we show that P(0) is true. 
That is, we show that the result is true of all elements specified to be in the set in the basis 
step of the definition. A consequence of the recursive step is that if we assume P(k) is true, 
it follows that P(k + 1) is true. When we have completed a proof using structural induction, 
we have shown that P(O) is true and that P(k) implies P(k + 1). By mathematical induction it 
follows that P(n) is true for all nonnegative integers n. This also shows that the result is true for 
all elements generated by the recursive definition, and shows that structural induction is a valid 
proof technique. 


EXAMPLES OF PROOFS USING STRUCTURAL INDUCTION Structural induction can 
be used to prove that all members of a set constructed recursively have a particular property. 
We will illustrate this idea by using structural induction to prove results about well-formed 
formulae, strings, and binary trees. For each proof, we have to carry out the appropriate basis 
step and the appropriate recursive step. For example, to use structural induction to prove a result 
about the set of well-formed formulae defined in Example 8, where we specify that T, F, and 
every propositional variable s are well-formed formulae and where we specify that if E and F 
are well-formed formulae, then (CE), (E A F), (E v F), (E > F), and (E © F) are well-formed 
formulae, we need to complete this basis step and this recursive step. 


BASIS STEP: Show that the result is true for T, F, and s whenever s is a propositional variable. 


RECURSIVE STEP: Show that if the result is true for the compound propositions p and q, it is 
also true for (~p), (p V q), (p A q), (p > q), and (p © q). 


Example 11 illustrates how we can prove results about well-formed formulae using struc- 
tural induction. 


Show that every well-formed formula for compound propositions, as defined in Example 8, 
contains an equal number of left and right parentheses. 


Solution: 


BASIS STEP: Each of the formula T, F, and s contains no parentheses, so clearly they contain 
an equal number of left and right parentheses. 


RECURSIVE STEP: Assume p and q are well-formed formulae, each containing an equal 
number of left and right parentheses. That is, if /, and /, are the number of left parentheses 
in p and q, respectively, and r, and r, are the number of right parentheses in p and q, respec- 
tively, then /, = r, and /, = r}. To complete the inductive step, we need to show that each of 
(=p), (p Vq), (p ^q), (p > q), and (p + q) also contains an equal number of left and right 


parentheses. The number of left parentheses in the first of these compound propositions equals 
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l, + 1 and in each of the other compound propositions equals }, + /, + 1. Similarly, the number 
of right parentheses in the first of these compound propositions equals 7, + 1 and in each of 
the other compound propositions equals r, + r} + 1. Because J, = r, and 1, = r}, it follows that 
each of these compound expressions contains the same number of left and right parentheses. 
This completes the proof by structural induction. < 


Suppose that P(w) is a propositional function over the set of strings w € &*. To use struc- 
tural induction to prove that P(w) holds for all strings w E€ &*, we need to complete both a basis 
step and a recursive step. These steps are: 


BASIS STEP: Show that P(A) is true. 


RECURSIVE STEP: Assume that P(w) is true, where w E€ &*. Show that if x € È, then P(wx) 
must also be true. 


Example 12 illustrates how structural induction can be used in proofs about strings. 


Use structural induction to prove that /(xy) = I(x) + L(y), where x and y belong to &*, the set of 
strings over the alphabet X. 


Solution: We will base our proof on the recursive definition of the set X* given in Definition 1 
and the definition of the length of a string in Example 7, which specifies that (A) = 0 and (wx) = 
(w) + 1 when w E€ &* and x E È. Let P(y) be the statement that (xy) = I(x) + L(y) whenever x 
belongs to &*. 


BASIS STEP: To complete the basis step, we must show that P(A) is true. That is, we must 
show that [(xA) = I(x) + L(A) for all x € &*. Because [(xA) = I(x) = I(x) + 0 = I(x) + [(A) for ev- 
ery string x, it follows that P(A) is true. 


RECURSIVE STEP: To complete the inductive step, we assume that P(y) is true and show 
that this implies that P(ya) is true whenever a € X. What we need to show is that /(xya) = 
I(x) + l(ya) for every a € ÈX. To show this, note that by the recursive definition of /(w) (given in 
Example 7), we have /(xya) = I(xy) + 1 and (ya) = L(y) + 1. And, by the inductive hypothesis, 
(xy) = I(x) + (y). We conclude that [(xya) = I(x) + I) + 1 = U(x) + lya). < 


We can prove results about trees or special classes of trees using structural induction. For 
example, to prove a result about full binary trees using structural induction we need to complete 
this basis step and this recursive step. 


BASIS STEP: Show that the result is true for the tree consisting of a single vertex. 


RECURSIVE STEP: Show that if the result is true for the full binary trees T, and T}, then it 
is true for tree T, - T, consisting of a root r, which has T, as its left subtree and T, as its right 
subtree. 


Before we provide an example showing how structural induction can be used to prove a 
result about full binary trees, we need some definitions. We will recursively define the height 
h(T) and the number of vertices n(T) of a full binary tree T. We begin by defining the height of 
a full binary tree. 


We define the height A(T) of a full binary tree T recursively. 
BASIS STEP: The height of the full binary tree T consisting of only a root r is h(T) = 0. 


RECURSIVE STEP: If T, and T, are full binary trees, then the full binary tree T = T, - T, 
has height A(T) = 1 + max(A(7,), A(T>)). 


THEOREM 2 
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If we let n(T) denote the number of vertices in a full binary tree, we observe that n(7) satisfies 
the following recursive formula: 


BASIS STEP: The number of vertices n(T) of the full binary tree T consisting of only a root r 
isn(T) = 1. 


RECURSIVE STEP: If T, and T, are full binary trees, then the number of vertices of the full 
binary tree T = T, - T, is n(T) = 1+n(T,) +n(7). 


We now show how structural induction can be used to prove a result about full binary trees. 


If T is a full binary tree T, then n(T) < 20+ — 1. 


Proof: We prove this inequality using structural induction. 


BASIS STEP: For the full binary tree consisting of just the root r the result is true because 
n(T) = 1 and A(T) = 0, so that n(T) = 1 < 2%! -1=1. 


RECURSIVE STEP: For the inductive hypothesis we assume that n(T,) < 27+! — 1 and 
n(T,) < 2'\7%)+! — 1 whenever T, and T, are full binary trees. By the recursive formulae for 


n(T) and A(T) we have n(T) = 1+ n(T,) + n(T,) and A(T) = 1 + max(h(T,), h(T,)). 
We find that 


nT) =14+n(7,)+n(L,) by the recursive formula for n(T) 


IH 
<1 +(2}7D+1 — 1) + (241 _ 1) by the inductive hypothesis 


< 2. max(2»T9)+1, 240)+1) — 1 because the sum of two terms is at most 2 
times the larger 
= 2 . Qmax(h(T).A(T,))+1 _ 4 because max(2*, 2”) = 274x017) 
a Ii by the recursive definition of h(T) 
= ATH _ 4. 
This completes the recursive step. < 


5.3.5 Generalized Induction 


We can extend mathematical induction to prove results about other sets that have the well- 
ordering property besides the set of integers. Although we will discuss this concept in detail in 
Section 9.6, we provide an example here to illustrate the usefulness of such an approach. 

As an example, note that we can define an ordering on N x N, the ordered pairs of non- 
negative integers, by specifying that (x,, y,) is less than or equal to (4, y2) if either x, < Xx, 
or x; =X, and y; < yz; this is called the lexicographic ordering. The set N x N with this 
ordering has the property that every subset of N x N has a least element (see Exercise 53 
in Section 9.6). This implies that we can recursively define the terms a,,,,, with m E€ N and 
n E N, and prove results about them using a variant of mathematical induction, as illustrated in 
Example 13. 
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Exercises 


1. Find f(1), f(2), f(3), and f(4) if f (n) is defined recursively 


is defined recursively for (m,n) E N x N by do = 0 and 


mn 


— Gntnt+1 ifn=Oandm > 0 
u amni +n ifn>0. 


Show that a,,,, = m +n(n + 1)/2 for all (m,n) E N X N, that is, for all pairs of nonnegative 
integers. 


Solution: We can prove that a„„ = m + n(n + 1)/2 using a generalized version of mathematical 
induction. The basis step requires that we show that this formula is valid when (m, n) = (0, 0). 
The induction step requires that we show that if the formula holds for all pairs smaller than 
(m, n) in the lexicographic ordering of N x N, then it also holds for (mm, n). 


BASIS STEP: Let (m, n) = (0, 0). Then by the basis case of the recursive definition of a,,,, we 


have ago = 0. Furthermore, when m = n = 0, m + n(n + 1)/2 = 0 + (0 ; 1)/2 = 0. This com- 
pletes the basis step. 


INDUCTIVE STEP: Suppose that ayy , =m +n'(n' + 1)/2 whenever (m, n’) is less than 
(m,n) in the lexicographic ordering of NxN. By the recursive definition, if n = 0, 
then amn = 4y—1,, + 1. Because (m -— 1,n) is smaller than (m,n), the inductive hypothe- 
sis tells us that am-in =m—1+n(n+1)/2, so that a,,,=m—1+n(n+1)/2+1=m+ 
n(n + 1)/2, giving us the desired equality. Now suppose that n > 0, SO amn = yy) +N. 
Because (m,n — 1) is smaller than (m, n), the inductive hypothesis tells us that a,,,,_) = m+ 
(n— 1)n/2, 80 amn =m + (n — 1)n/2+n =m + (n? —n +2n)/2 = m + n(n + 1)/2. This fin- 
ishes the inductive step. < 


As mentioned, we will justify this proof technique in Section 9.6. 


5. Determine whether each of these proposed definitions is 


by f(0) = 1 and for n = 0, 1, 2,... 


a) fn+1) =f(™ +2. 
b) f(n + 1) = 3f(n). 
ce) fn+ 1) =y”. 
d) fn+) =f} +f) +1. 
. Find f(1), f(2), f(3), f(4, and f(5) if f(n) is defined re- 
cursively by f(0) = 3 and for n = 0, 1, 2, ... 
a) f(n + 1) = -2f (n). 
b) f(n + 1) = 3f() +7. 
© f(n+1) =f? — 2f(n) -2. 
d) f(n + 1) = oA, 
. Find f(2), f (3), f(4), and f(5) if f is defined recursively 
by f(0) = —1, fC) = 2, and for n = 1,2, ... 
a) f(n+ 1) =f(n)+3f(n — 1). 
b) f(n + 1) =fayf(n— 1). 
©) f(n+ 1) =3fnf —4f(n— 1}. 
d) f(n +1) =f- 1)/f). 
. Find f(2), f(3), f(4), and f(5) if f is defined recursively 
by f(0) = f(1) = 1 and for n = 1,2, ... 


a) f(n+1)=fn)- f(n- 1). 
b) f(n+1)=fnf(n-— 1). 

©) f(n+1) =f} +fn- 1%. 
d) fín + 1) =f(@)/f(n- 1). 


a valid recursive definition of a function f from the set 

of nonnegative integers to the set of integers. If f is well 

defined, find a formula for f(n) when n is a nonnegative 

integer and prove that your formula is valid. 

a) f(0) = 0, f(n) =2f(n-—2)forn > 1 

b) f0) = 1, f(n) =f(n— 1)—1forn> 1 

c) f0) = 2, f(1) =3,f(n)=f(n—1)—1forn > 2 

d) f(0) = 1, fC) = 2, f(n) = 2f (n — 2) for n > 2 

e) f(0) = 1, f(n) = 3f(n — 1) if n is odd and n > 1 and 
f(n) = 9f(n — 2) ifn is even and n > 2 


. Determine whether each of these proposed definitions is 


a valid recursive definition of a function f from the set 

of nonnegative integers to the set of integers. If f is well 

defined, find a formula for f(n) when n is a nonnegative 

integer and prove that your formula is valid. 

a) f(0) =1,f(™) = -f(n — l) forn > 1 

b) fO) = 1, fd) = 0, fQ) = 2, fn) = fn- 3) 
forn > 3 

c) f0) =0, fC) = 1, f(@) = 2f(n + 1) forn > 2 

d) f(0) =0, fC) = 1, f(n) = 2f(n — 1) forn > 1 

e) f(0) = 2, f(n) =f(n — 1) if n is odd and n> 1 and 
fm) = 2f(n —2) ifn > 2 


10. 


11. 


. Give a recursive definition of the sequence {a,}, n = 


1,2, 3, 2. tf 
a) a, = 6n. b) a, =2n+1. 
c) a, = 10". d) a, =5. 
. Give a recursive definition of the sequence {a,}, n = 
1,253) 00¢8 
a) a, =4n—2. b) a, =1+(-1)". 


c) a, =n(n+ 1). d) a, =n’. 


. Let F be the function such that F(n) is the sum of the first 


n positive integers. Give a recursive definition of F(n). 
Give a recursive definition of S,,(m), the sum of the inte- 
ger m and the nonnegative integer n. 

Give a recursive definition of P„(n), the product of the 
integer m and the nonnegative integer n. 


In Exercises 12-19 f, is the nth Fibonacci number. 


12. 


13. 


*14, 


* 15, 


* 16. 


17. 


18. 


19. 


* 20. 


*21. 


22. 


Prove that f? +f +++ +f? = fifi, when n is a positive 
integer. 
Prove that fi +f; + + +fon-1 = Jon When n is a positive 
integer. 
Show that f,.if,-1 — f = (—1)” when n is a positive in- 
teger. 

2 . 
Show that Soh +a + + hin-vhin = f3, When n is a 
positive integer. 
Show that Jo -fi +h Paa = fon- Ffon = font =1 
when n7 is a positive integer. 
Determine the number of divisions used by the Euclidean 
algorithm to find the greatest common divisor of the Fi- 
bonacci numbers f, and fı, where n is a nonnegative in- 
teger. Verify your answer using mathematical induction. 
Let 


Show that 


ni nii Sn 
ag Ta Fa 


when n7 is a positive integer. 

By taking determinants of both sides of the equation in 
Exercise 18, prove the identity given in Exercise 14. (Re- 
j i is ad — bc.) 
Give a recursive definition of the functions max and min 
so that max(d,, æ, ...,a,) and min(ay, dy, ..., a„) are the 
maximum and minimum of the n numbers a}, do, ..., Ap, 
respectively. 


call that the determinant of the matrix 


Let a), d5,...,4,, and b}, by, ..., b, be real numbers. Use 
the recursive definitions that you gave in Exercise 20 to 


prove these. 


a) max(—d,, —d),..., —a,) = — Mindy, ay, ..., Ap) 
b) max(a, + b, a, + bz, ..., ap + bp) 

< max(dj, do, ..., 4„) + Max (b, bz, ..., bp) 
c) min(a, + by, a + bz, ..., a, + bp) 

> min(a;, a, ...,4,,) + min(d,, by, ..., b,) 


Show that the set S defined by 1 € Sands + t € S when- 
ever s € S and t € S is the set of positive integers. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 
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Give a recursive definition of the set of positive integers 
that are multiples of 5. 

Give a recursive definition of 

a) the set of odd positive integers. 

b) the set of positive integer powers of 3. 

c) the set of polynomials with integer coefficients. 
Give a recursive definition of 

a) the set of even integers. 

b) the set of positive integers congruent to 2 modulo 3. 
c) the set of positive integers not divisible by 5. 

Let S be the set of positive integers defined by 


Basis step: 1 €S. 

Recursive step: If n € S, then 3n +2 E€ S and n? E S. 

a) Show that if n € S, then n = 1 (mod 4). 

b) Show that there exists an integer m = 1 (mod 4) that 
does not belong to S. 

Let S be the set of positive integers defined by 


Basis step: 5 E€ S. 

Recursive step: If n € S, then 3n € S and n? E S. 

a) Show that ifn E€ S, then n = 5 (mod 10). 

b) Show that there exists an integer m = 5 (mod 10) that 
does not belong to S. 

Let S be the subset of the set of ordered pairs of integers 

defined recursively by 


Basis step: (0,0) E S. 


Recursive step: If (a, b) € S, then (a + 2, b + 3) € S and 

(a+3,b+2) ES. 

a) List the elements of S produced by the first five appli- 
cations of the recursive definition. 

b) Use strong induction on the number of applications 
of the recursive step of the definition to show that 
5|a+b when (a, b) E€ S. 

c) Use structural induction to show that 5 | a + b when 
(a,b) ES. 

Let S be the subset of the set of ordered pairs of integers 

defined recursively by 


Basis step: (0,0) E S. 


Recursive step: If (a,b)EsS, then (a,b+1)E€ 8S, 

(a+1,b+1)E€8sS,and(a+2,b+1)Es. 

a) List the elements of S produced by the first four ap- 
plications of the recursive definition. 

b) Use strong induction on the number of applications of 
the recursive step of the definition to show that a < 2b 
whenever (a, b) € S. 

c) Use structural induction to show that a < 2b when- 
ever (a, b) E S. 

Give a recursive definition of each of these sets of ordered 

pairs of positive integers. [Hint: Plot the points in the set in 

the plane and look for lines containing points in the set.] 

a) S= {(a, b) |a E Z*, b € Z*, and a + b is odd} 

b) S= {(a,b)|a E Z*, b E€ Z*, anda |b} 

c) S= {(a,b)|a E Z*,b € Z*, and 3|a +b} 

Give a recursive definition of each of these sets of or- 

dered pairs of positive integers. Use structural induction 
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to prove that the recursive definition you found is correct. 
[Hint: To find a recursive definition, plot the points in the 
set in the plane and look for patterns. ] 


a) S= {(a, b) |a E€ Zt, b € Zt, and a + b is even} 

b) S= {(a, b) |a € Z*, b € Z*, and a or b is odd} 

c) S= {(a, b) |a E€ Z*, b € Z*,a + bis odd, and 3 | b} 
Prove that in a bit string, the string 01 occurs at most one 
more time than the string 10. 


32. 


33. 


Define well-formed formulae of sets, variables represent- 
ing sets, and operators from { , U, N, —}. 


34. a) Give a recursive definition of the function ones(s), 


which counts the number of ones in a bit string s. 
b) Use structural induction to prove that ones(st) = 
ones(s) + ones(t). 


35. a) Give a recursive definition of the function m(s), which 
equals the smallest digit in a nonempty string of dec- 
imal digits. 

b) Use structural induction to prove that m(st) = 
min(m(s), m(t)). 

The reversal of a string is the string consisting of the symbols 

of the string in reverse order. The reversal of the string w is 

denoted by w”. 


36. Find the reversal of the following bit strings. 


a) 0101 b) 11011 c) 1000 10010111 


Give a recursive definition of the reversal of a string. 
[Hint: First define the reversal of the empty string. Then 
write a string w of length n + 1 as xy, where x is a string 
of length n, and express the reversal of w in terms of x? 
and y.] 


37. 


= wrk, 


*38. Ryt 


39. 


Use structural induction to prove that (w; w3)? 


Give a recursive definition of w’, where w is a string and i 
is a nonnegative integer. (Here w’ represents the concate- 
nation of i copies of the string w.) 


*40. Give a recursive definition of the set of bit strings that are 


palindromes. 


41. When does a string belong to the set A of bit strings de- 


fined recursively by 


AEA 
Oxl EAifx EA, 


where À is the empty string? 


* 42. Recursively define the set of bit strings that have more 


zeros than ones. 


43. Use Exercise 39 and mathematical induction to show that 
I(w') = i - I(w), where w is a string and i is a nonnegative 


integer. 


*44. Show that (w)! = (w!)® whenever w is a string and i is 
a nonnegative integer; that is, show that the ith power of 
the reversal of a string is the reversal of the ith power of 


the string. 


Use structural induction to show that n(T) > 2h(T) + 1, 
where T is a full binary tree, n(T) equals the number of 
vertices of T, and A(T) is the height of T. 


45. 


* 49, 


Links) 


The set of leaves and the set of internal vertices of a full bi- 
nary tree can be defined recursively. 


Basis step: The root r is a leaf of the full binary tree with 
exactly one vertex r. This tree has no internal vertices. 


Recursive step: The set of leaves of the tree T = T, - T, is 
the union of the sets of leaves of T) and of T,. The inter- 
nal vertices of T are the root r of T and the union of the 
set of internal vertices of T} and the set of internal vertices 
of T}. 

46. Use structural induction to show that (T), the number 
of leaves of a full binary tree T, is 1 more than i(T), the 
number of internal vertices of T. 

Use generalized induction as was done in Example 13 to 
show that if a,,,, is defined recursively by ag = 0 and 


mn 
ann = { 
then a 


mn = mM +n for all (m,n) E NXN. 
Use generalized induction as was done in Example 13 to 
show that if a,,,, is defined recursively by a,, = 5 and 


ann = { 
then a 


mn = 20m +n) + 1 for all (m,n) € Z+ x ZF. 
A partition of a positive integer n is a way to write n as 
a sum of positive integers where the order of terms in the 
sum does not matter. For instance, 7 =3+2+1+1lisa 
partition of 7. Let P,,, equal the number of different parti- 
tions of m, and let P,,,,, be the number of different ways to 
express m as the sum of positive integers not exceeding n. 


a) Show that P, =P 


m,m m* 


b) Show that the following recursive definition for P 


47. 


Am-1,n F 1 if n= 0 and m> 0 
Am,n-1 + 1 if n> 0, 


48. 


Ann +2 ifn=landm> 1 
Amn-1 + 2 ifn > 1, 


m,n 


is correct: 
1 ifm=1 
1 ifn= 1 
Eam l i ifm<n 
LEP ami ifm=n> 1 
Pari th, ifm>n> 1. 


m—-n,n 


c) Find the number of partitions of 5 and of 6 using this 
recursive definition. 


Consider the following inductive definition of a version 
of Ackermann’s function. This function was named after 
Wilhelm Ackermann, a German mathematician who was a 
student of the great mathematician David Hilbert. Acker- 
mann’s function plays an important role in the theory of re- 
cursive functions and in the study of the complexity of certain 
algorithms involving set unions. (There are several different 
variants of this function. All are called Ackermann’s function 
and have similar properties even though their values do not 
always agree.) 


2n ifm=0 

0 ifm > l andn =0 
A(m, n) = . 

2 ifm > l andn = 1 


A(m—1,A(m,n—1)) ifm> landn>2 


Exercises 50-57 involve this version of Ackermann’s func- 
tion. 
50. Find these values of Ackermann’s function. 
a) ACI, 0) b) A(0, 1) 
c) AQ, 1) d) A(2, 2) 
51. Show that A(m, 2) = 4 whenever m > 1. 
52. Show that A(1, n) = 2” whenever n > 1. 
53. Find these values of Ackermann’s function. 
a) A(2, 3) *b) A(3, 3) 
* 54. Find A(3, 4). 


** 55, Prove that A(m, n + 1) > A(m, n) whenever m and n are 


nonnegative integers. 
* 56. Prove that A(m + 1, n) > A(m, n) whenever m and n are 
nonnegative integers. 

57. Prove that A(i, j) > j whenever i and j are nonnegative in- 
tegers. 

58. Use mathematical induction to prove that a function F de- 
fined by specifying F(0) and a rule for obtaining F(n + 1) 
from F(n) is well defined. 

59. Use strong induction to prove that a function F defined by 
specifying F(0) and a rule for obtaining F(n + 1) from 
the values F(k) for k = 0, 1, 2, ..., n is well defined. 

60. Show that each of these proposed recursive definitions of 
a function on the set of positive integers does not produce 
a well-defined function. 

a) F(n)= 1 + F(|n/2]) forn > 1 and F(1) = 1. 

b) Fm) =1+F-3) for n22, F(1)=2, and 
F(2) = 3. 

ce) F(n) = 1 + F(n/2) forn > 2, F(1) = 1, and F(2) = 2. 

d) F(n) = 1 + F(n/2) if nis even andn > 2, F(n) = 1 — 
F(n — 1) if nis odd, and F(1) = 1. 

e) F(n)= 1 + F(n/2) if n is even and n > 2, F(n) = 
F(3n — 1) if nis odd and n > 3, and F(1) = 1. 

61. Show that each of these proposed recursive definitions of 
a function on the set of positive integers does not produce 
a well-defined function. 


a) F(n) = 14+ F(|(n + 1)/2]) for n>1 and 
FI) =1. 

b) ieee ee 

c) F(n) = 1 + F(n/3) forn > 3, F(1) = 1, F(2) = 2, and 
F@) =3. 

d) F(n) = 1 + F(n/2) if nis even andn > 2, F(n) = 1 + 
F(n — 2) if n is odd, and F(1) = 1. 

e) F(n)= 1 + F(F(n — 1)) ifn > 2 and F(1) = 2. 
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Exercises 62—64 deal with iterations of the logarithm func- 
tion. Let log n denote the logarithm ofn to the base 2, as usual. 
The function log™ n is defined recursively by 


n ifk=0 
w _ Jlogdog*-) n) if log) n is defined 
8 n= and positive 


undefined otherwise. 


The iterated logarithm is the function log* n whose value at n 

is the smallest nonnegative integer k such that log n < 1. 

62. Find these values. 
a) log” 16 

(3) 765536 


b) log® 256 
c) log d) log 2°” 
63. Find the value of log* n for these values of n. 


a) 2 b) 4 c) 8 d) 16 
e) 256 f) 65536 g) 27048 


64. Find the largest integer n such that log* n = 5. Determine 
the number of decimal digits in this number. 
Exercises 65-67 deal with values of iterated functions. Sup- 
pose that f(n) is a function from the set of real numbers, or 
positive real numbers, or some other set of real numbers, to 
the set of real numbers such that f(n) is monotonically increas- 
ing [that is, f(n) < f(m) when n < m) and f(n) < n for all n in 
the domain of f.] The function f® (n) is defined recursively by 


on fn ifk=0 
f (n) = oe ifk> 0. 


Furthermore, let c be a positive real number. The iterated 
function f* is the number of iterations of f required to reduce 
its argument to c or less, so f*(7) is the smallest nonnegative 
integer k such that f*(n) < c. 

65. Let f(n) = n — a, where a is a positive integer. Find a for- 


mula for f(n). What is the value of f*(n) when n is a 
positive integer? 
66. Let f(n) = n/2. Find a formula for f(n). What is the 


value of ff (n) when n is a positive integer? 


67. Let f(n) = y/n. Find a formula for f® (n). What is the 
value of fž(n) when n is a positive integer? 


Sometimes we can reduce the solution to a problem with a particular set of input values to the 


Here’s a famous 
humorous quote: “To 
understand recursion, 
you must first 
understand recursion.” 


solution of the same problem with smaller input values. For instance, the problem of finding 
the greatest common divisor of two positive integers a and b, where b > a, can be reduced 
to finding the greatest common divisor of a pair of smaller integers, namely, b moda and a, 
because gcd(b mod a, a) = gcd(a, b). 


When such a reduction can be done, the solution to the original problem can be found with 
a sequence of reductions, until the problem has been reduced to some initial case for which the 
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Definition 1 


Links » 


EXAMPLE 1 


Extra » 
Examples 


EXAMPLE 2 


solution is known. For instance, for finding the greatest common divisor, the reduction continues 
until the smaller of the two numbers is zero, because gcd(a, 0) = a when a > 0. 

We will see that algorithms that successively reduce a problem to the same problem with 
smaller input are used to solve a wide variety of problems. 


An algorithm is called recursive if it solves a problem by reducing it to an instance of the 
same problem with smaller input. 


We will describe a variety of different recursive algorithms in this section. 


Give a recursive algorithm for computing n!, where n is a nonnegative integer. 


Solution: We can build a recursive algorithm that finds n!, where n is a nonnegative integer, 
based on the recursive definition of n!, which specifies that n! = n - (n — 1)! when n is a positive 
integer, and that 0! = 1. To find n! for a particular integer n, we use the recursive step n times, 
each time replacing a value of the factorial function with the value of the factorial function at 
the next smaller integer. At this last step, we insert the value of 0!. The recursive algorithm we 
obtain is displayed as Algorithm 1. 

To help understand how this algorithm works, we trace the steps used by the algorithm 
to compute 4!. First, we use the recursive step to write 4! = 4- 3!. We then use the recursive 
step repeatedly to write 3! = 3-2!,2!=2-1!, and 1! =1-0!. Inserting the value of 0! = 1, 
and working back through the steps, we see that 1!=1-1=1,2!=2-1!=2,3!=3-2!= 
3-2=6,and4!=4-3!=4-6=24. < 


ALGORITHM 1 A Recursive Algorithm for Computing n!. 


procedure factorial(n: nonnegative integer) 
if n = 0 then return 1 

else return z - factorial(n — 1) 

{output is n!} 


Example 2 shows how a recursive algorithm can be constructed to evaluate a function from its 
recursive definition. 


Give a recursive algorithm for computing a”, where a is a nonzero real number and n is a 
nonnegative integer. 


Solution: We can base a recursive algorithm on the recursive definition of a”. This definition 
states that a”*! = a- a" for n > 0 and the initial condition a? = 1. To find a”, successively use 
the recursive step to reduce the exponent until it becomes zero. We give this procedure in 
Algorithm 2. < 


ALGORITHM 2 A Recursive Algorithm for Computing a”. 


procedure power(a: nonzero real number, n: nonnegative integer) 


if n = 0 then return 1 
else return a - power(a, n — 1) 


{output is a” } 


EXAMPLE 3 


EXAMPLE 4 
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Next we give a recursive algorithm for finding greatest common divisors. 


Give a recursive algorithm for computing the greatest common divisor of two nonnegative in- 
tegers a and b witha < b. 


Solution: We can base a recursive algorithm on the reduction gcd(a, b) = gcd(b mod a, a) and 
the condition gcd(0, b) = b when b > 0. This produces the procedure in Algorithm 3, which is 
a recursive version of the Euclidean algorithm. 

We illustrate the workings of Algorithm 3 with a trace when the input is a = 5, b= 8. 
With this input, the algorithm uses the “else” clause to find that gcd(5, 8) = gcd(8 mod 5,5) = 
gcd(3, 5). It uses this clause again to find that gcd(3, 5) = gcd(5 mod 3,3) = gcd(2, 3), then 
to get gcd(2, 3) = gcd(3 mod 2,2) = gcd(1, 2), then to get gcd(1, 2) = gcd(2 mod 1, 1) = 
gcd(0, 1). Finally, to find gcd(0, 1) it uses the first step with a = 0 to find that gcd(O, 1) = 1. 
Consequently, the algorithm finds that gcd(5, 8) = 1. < 


ALGORITHM 3 A Recursive Algorithm for Computing gcd(a, b). 


procedure gcd(a, b: nonnegative integers with a < b) 
if a = 0 then return b 

else return gcd(b mod a, a) 

{output is gcd(a, b)} 


Devise a recursive algorithm for computing b” mod m, where b, n, and m are integers with 
m>2,n>0, andl<b<™m. 


Solution: We can base a recursive algorithm on the fact that 
b” mod m = (b - (b"-! mod m)) mod m, 


which follows by Corollary 2 in Section 4.1, and the initial condition b? mod m = 1. We leave 
this as Exercise 12 for the reader. 

However, we can devise a much more efficient recursive algorithm based on the observation 
that 


b” mod m = (b" mod m)? mod m 
when n is even and 
b” mod m = ((b™"/! mod m)? mod m - b mod m) mod m 


when n is odd, which we describe in pseudocode as Algorithm 4. 

We trace the execution of Algorithm 4 with input b = 2, n = 5, and m = 3 to illustrate how 
it works. First, because n = 5 is odd we use the “else” clause to see that mpower(2, 5, 3) = 
(mpower(2, 2,3) mod 3 - 2 mod 3) mod 3. We next use the “else if” clause to see that 
mpower(2, 2, 3) = mpower(2, 1, 3) mod 3. Using the “else” clause again, we see that 
mpower(2, 1, 3) = (mpower(2, 0,3)? mod 3 - 2 mod 3) mod 3. Finally, using the “if” clause, 
we see that mpower(2, 0, 3) = 1. Working backwards, it follows that mpower(2, 1, 3) = 
(1? mod 3 - 2 mod 3) mod 3 = 2, so mpower(2, 2, 3) = 2? mod 3 = 1, and finally 
mpower(2, 5, 3) = (1? mod 3-2 mod 3) mod 3 = 2. < 
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EXAMPLE 5 


EXAMPLE 6 


ALGORITHM 4 Recursive Modular Exponentiation. 


procedure mpower(b, n, m: integers with b > 0 and m > 2, n > 0) 
if n = 0 then 

return 1 
else if n is even then 

return mpower(b, n/2, m) mod m 
else 

return (mpower(b, |n/2], m) mod m - b mod m) mod m 
{output is b” mod m} 


We will now give recursive versions of searching algorithms that were introduced in Section 3.1. 


Express the linear search algorithm as a recursive procedure. 


Solution: To search for the first occurrence of x in the sequence a}, az, ..., a, at the ith step of 
the algorithm, x and a; are compared. If x equals a;, then the algorithm returns i, the location of 
x in the sequence. Otherwise, the search for the first occurrence of x is reduced to a search in 
a sequence with one fewer element, namely, the sequence a,,j, ...,d,. The algorithm returns 0 
when x is never found in the sequence after all terms have been examined. We can now give a 
recursive procedure, which is displayed as pseudocode in Algorithm 5. 

Let search (i, j, x) be the procedure that searches for the first occurrence of x in the sequence 
ai lizi -> Aj. The input to the procedure consists of the triple (1, n, x). The algorithm terminates 
at a step if the first term of the remaining sequence is x or if there is only one term of the sequence 
and this is not x. If x is not the first term and there are additional terms, the same procedure is 
carried out but with a search sequence of one fewer term, obtained by deleting the first term 
of the search sequence. If the algorithm terminates without x having been found, the algorithm 
returns the value 0. < 


ALGORITHM 5 A Recursive Linear Search Algorithm. 


procedure search(i, j, x: integers, 1 <i <j <n) 
if a; = x then 
return i 
else if i = j then 
return 0 
else 
return search(i + 1, j, x) 
{output is the location of x in a), a, ..., a, if it appears; otherwise it is 0} 


Construct a recursive version of a binary search algorithm. 


Solution: Suppose we want to locate x in the sequence a,, az, ..., a„ of integers in increasing 
order. To perform a binary search, we begin by comparing x with the middle term, 4) (,,41)/2)- 
Our algorithm will terminate if x equals this term and return the location of this term in the 
sequence. Otherwise, we reduce the search to a smaller search sequence, namely, the first half 
of the sequence if x is smaller than the middle term of the original sequence, and the second 


EXAMPLE 7 


EXAMPLE 8 
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half otherwise. We have reduced the solution of the search problem to the solution of the same 
problem with a sequence at most half as long. If we have never encountered the search term x, 
our algorithm returns the value 0. We express this recursive version of a binary search algorithm 
as Algorithm 6. < 


ALGORITHM 6 A Recursive Binary Search Algorithm. 


procedure binary search(i, j, x: integers, 1 < i < j <n) 
m := |(i + j)/2] 
if x = a„ then 
return m 
else if (x < a,, and i < m) then 
return binary search(i, m — 1, x) 
else if (x > a,, and j > m) then 
return binary search(m + 1, j, x) 
else return 0 
{output is location of x in a}, a, ..., a,, if it appears; otherwise it is 0} 


5.4.2 Proving Recursive Algorithms Correct 


Mathematical induction, and its variant strong induction, can be used to prove that a recursive 
algorithm is correct, that is, that it produces the desired output for all possible input values. 
Examples 7 and 8 illustrate how mathematical induction or strong induction can be used to 
prove that recursive algorithms are correct. First, we will show that Algorithm 2 is correct. 


Prove that Algorithm 2, which computes powers of real numbers, is correct. 


Solution: We use mathematical induction on the exponent n. 


BASIS STEP: If n = 0, the first step of the algorithm tells us that power (a, 0) = 1. This is correct 
because a? = 1 for every nonzero real number a. This completes the basis step. 


INDUCTIVE STEP: The inductive hypothesis is the statement that power (a, k) = a* for all 
a + 0 for an arbitrary nonnegative integer k. That is, the inductive hypothesis is the statement 
that the algorithm correctly computes a‘. To complete the inductive step, we show that if the 
inductive hypothesis is true, then the algorithm correctly computes a‘t!, Because k + 1 is a 
positive integer, when the algorithm computes a‘t!, the algorithm sets power (a, k + 1) = 
a: power (a, k). By the inductive hypothesis, we have power (a, k) = a‘, so power (a, k + 1) = 
a: power (a, k) = a- a‘ = a**', This completes the inductive step. 


We have completed the basis step and the inductive step, so we can conclude that Algorithm 
2 always computes a” correctly when a ¥ 0 and n is a nonnegative integer. < 


Generally, we need to use strong induction to prove that recursive algorithms are correct, 
rather than just mathematical induction. Example 8 illustrates this; it shows how strong induc- 
tion can be used to prove that Algorithm 4 is correct. 


Prove that Algorithm 4, which computes modular powers, is correct. 


Solution: We use strong induction on the exponent n. 
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BASIS STEP: Let b be an integer and m an integer with m > 2. When n = 0, the algorithm sets 
mpower(b, n, m) equal to 1. This is correct because b? mod m = 1. The basis step is complete. 


INDUCTIVE STEP: For the inductive hypothesis we assume that mpower(b, j, m) = b’ mod m 
for all integers 0 < j < k whenever b is a positive integer and m is an integer with m > 2. 
To complete the inductive step, we show that if the inductive hypothesis is correct, then 
mpower(b, k, m) = bt mod m. Because the recursive algorithm handles odd and even values 
of k differently, we split the inductive step into two cases. 

When k is even, we have 


mpower(b, k, m) = (mpower(b, k/2, m))} mod m = (b*/? mod m)? mod m = b* mod m, 


where we have used the inductive hypothesis to replace mpower(b, k/2, m) by b*/? mod m. 
When k is odd, we have 


mpower(b, k, m) = ((mpower(b, |k/2], m)? mod m -b mod m) mod m 
= ((b|*/?! mod m)? mod m - b mod m) mod m 


= b7K/21+! mod m= b* mod m, 


using Corollary 2 in Section 4.1, because 2[k/2] + 1 = 2(k — 1)/2 + 1 =k when k is odd. Here 
we have used the inductive hypothesis to replace mpower(b, |k/2], m) by b\“/?! mod m. This 
completes the inductive step. 


We have completed the basis step and the inductive step, so by strong induction we know 
that Algorithm 4 is correct. < 


5.4.3 Recursion and Iteration 


A recursive definition expresses the value of a function at a positive integer in terms of the val- 
ues of the function at smaller integers. This means that we can devise a recursive algorithm to 
evaluate a recursively defined function at a positive integer. Instead of successively reducing 
the computation to the evaluation of the function at smaller integers, we can start with the value 
of the function at one or more integers, the base cases, and successively apply the recursive 
definition to find the values of the function at successive larger integers. Such a procedure is 
called iterative. Often an iterative approach for the evaluation of a recursively defined sequence 
requires much less computation than a procedure using recursion (unless special-purpose recur- 
sive machines are used). This is illustrated by the iterative and recursive procedures for finding 
the nth Fibonacci number. The recursive procedure is given first. 


ALGORITHM 7 A Recursive Algorithm for Fibonacci Numbers. 


procedure fibonacci(n: nonnegative integer) 

if n = 0 then return 0 

else if n = 1 then return 1 

else return fibonacci(n — 1) + fibonacci(n — 2) 
{output is fibonacci(n)} 


When we use a recursive procedure to find f,, we first express f, as f,_; + f,-2. Then we replace 
both of these Fibonacci numbers by the sum of two previous Fibonacci numbers, and so on. 
When f, or fọ arises, it is replaced by its value. 
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FIGURE 1 Evaluating f, recursively. 


Note that at each stage of the recursion, until f, or fọ is obtained, the number of Fibonacci 
numbers to be evaluated has doubled. For instance, when we find f} using this recursive algo- 
rithm, we must carry out all the computations illustrated in the tree diagram in Figure 1. This 
tree consists of a root labeled with f}, and branches from the root to vertices labeled with the 
two Fibonacci numbers f} and f that occur in the reduction of the computation of f,. Each sub- 
sequent reduction produces two branches in the tree. This branching ends when fọ and f, are 
reached. The reader can verify that this algorithm requires f,,, — 1 additions to find f. 

Now consider the amount of computation required to find f, using the iterative approach in 
Algorithm 8. 


ALGORITHM 8 An Iterative Algorithm for Computing Fibonacci Numbers. 


procedure iterative fibonacci(n: nonnegative integer) 
if n = 0 then return 0 


else 

x= 0 

y:=1 

for i := 1 ton— 1 
z:=x+y 
x= y 
y:= 

return y 


{output is the nth Fibonacci number} 


This procedure initializes x as fọ = 0 and y as f} = 1. When the loop is traversed, the sum of x 
and y is assigned to the auxiliary variable z. Then x is assigned the value of y and y is assigned 
the value of the auxiliary variable z. Therefore, after going through the loop the first time, it 
follows that x equals f; and y equals fọ + fı = f2. Furthermore, after going through the loop 
n — | times, x equals f,_; and y equals f, (the reader should verify this statement). Only n — 1 
additions have been used to find f, with this iterative approach when n > 1. Consequently, this 
algorithm requires far less computation than does the recursive algorithm. 

We have shown that a recursive algorithm may require far more computation than an iter- 
ative one when a recursively defined function is evaluated. It is sometimes preferable to use a 
recursive procedure even if it is less efficient than the iterative procedure. In particular, this is 
true when the recursive approach is easily implemented and the iterative approach is not. (Also, 
machines designed to handle recursion may be available that eliminate the advantage of using 
iteration.) 
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FIGURE2_ The merge sort of 8, 2, 4, 6, 9, 7, 10, 1, 5, 3. 


5.4.4 The Merge Sort 


We now describe a recursive sorting algorithm called the merge sort algorithm. We will demon- 
strate how the merge sort algorithm works with an example before describing it in generality. 


Use the merge sort to put the terms of the list 8, 2, 4, 6, 9, 7, 10, 1, 5, 3 in increasing order. 


Solution: A merge sort begins by splitting the list into individual elements by successively split- 
ting lists in two. The progression of sublists for this example is represented with the balanced 
binary tree of height 4 shown in the upper half of Figure 2. 

Sorting is done by successively merging pairs of lists. At the first stage, pairs of individual 
elements are merged into lists of length two in increasing order. Then successive merges of 
pairs of lists are performed until the entire list is put into increasing order. The succession of 
merged lists in increasing order is represented by the balanced binary tree of height 4 shown in 
the lower half of Figure 2 (note that this tree is displayed “upside down’). < 


In general, a merge sort proceeds by iteratively splitting lists into two sublists of equal 
length (or where one sublist has one more element than the other) until each sublist contains one 
element. This succession of sublists can be represented by a balanced binary tree. The procedure 
continues by successively merging pairs of lists, where both lists are in increasing order, into a 
larger list with elements in increasing order, until the original list is put into increasing order. 
The succession of merged lists can be represented by a balanced binary tree. 

We can also describe the merge sort recursively. To do a merge sort, we split a list into 
two sublists of equal, or approximately equal, size, sorting each sublist using the merge sort 
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algorithm, and then merging the two lists. The recursive version of the merge sort is given in 
Algorithm 9. This algorithm uses the subroutine merge, which is described in Algorithm 10. 


ALGORITHM 9 A Recursive Merge Sort. 


procedure mergesort(L = ay, ..., d,,) 
ifn > 1 then 

m := |n/2| 

Ly := Gy, Ay, <- , Gy 

L, = Amt Am4 +o Ay 


L := merge(mergesort(L,), mergesort(L,)) 
{L is now sorted into elements in nondecreasing order} 


An efficient algorithm for merging two ordered lists into a larger ordered list is needed to 
implement the merge sort. We will now describe such a procedure. 


Merge the two lists 2, 3, 5, 6 and 1, 4. 


Solution: Table 1 illustrates the steps we use. First, compare the smallest elements in the two 
lists, 2 and 1, respectively. Because 1 is the smaller, put it at the beginning of the merged list 
and remove it from the second list. At this stage, the first list is 2, 3, 5, 6, the second is 4, and 
the combined list is 1. 

Next, compare 2 and 4, the smallest elements of the two lists. Because 2 is the smaller, add 
it to the combined list and remove it from the first list. At this stage the first list is 3, 5, 6, the 
second is 4, and the combined list is 1, 2. 

Continue by comparing 3 and 4, the smallest elements of their respective lists. Because 3 
is the smaller of these two elements, add it to the combined list and remove it from the first list. 
At this stage the first list is 5, 6, and the second is 4. The combined list is 1, 2, 3. 

Then compare 5 and 4, the smallest elements in the two lists. Because 4 is the smaller of 
these two elements, add it to the combined list and remove it from the second list. At this stage 
the first list is 5, 6, the second list is empty, and the combined list is 1, 2, 3, 4. 

Finally, because the second list is empty, all elements of the first list can be appended to 
the end of the combined list in the order they occur in the first list. This produces the ordered 
list 1, 2, 3, 4, 5, 6. < 


We will now consider the general problem of merging two ordered lists L} and L, into an 
ordered list L. We will describe an algorithm for solving this problem. Start with an empty 
list L. Compare the smallest elements of the two lists. Put the smaller of these two elements 


TABLE 1 Merging the Two Sorted Lists 2, 3, 5, 6 and 1, 4. 


14 1<2 
1 2<4 


12 3<4 
123 4<5 
1234 

123456 
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LEMMA 1 


at the right end of L, and remove it from the list it was in. Next, if one of L, and L, is empty, 
append the other (nonempty) list to L, which completes the merging. If neither L, nor L, is 
empty, repeat this process. Algorithm 10 gives a pseudocode description of this procedure. 

We will need estimates for the number of comparisons used to merge two ordered lists in 
the analysis of the merge sort. We can easily obtain such an estimate for Algorithm 10. Each 
time a comparison of an element from L, and an element from L, is made, an additional element 
is added to the merged list L. However, when either L, or L, is empty, no more comparisons 
are needed. Hence, Algorithm 10 is least efficient when m + n — 2 comparisons are carried out, 
where m and n are the number of elements in L, and L,, respectively, leaving one element in 
each of L, and L,. The next comparison will be the last one needed, because it will make one 
of these lists empty. Hence, Algorithm 10 uses no more than m + n — 1 comparisons. Lemma 1 
summarizes this estimate. 


ALGORITHM 10 Merging Two Lists. 


procedure merge(L,, L,: sorted lists) 
L := empty list 
while L, and L, are both nonempty 
remove smaller of first elements of L, and L, from its list; put it at the right end of L 
if this removal makes one list empty then remove all elements from the other list and 
append them to L 
return L{L is the merged list with elements in increasing order} 


Two sorted lists with m elements and n elements can be merged into a sorted list using no 
more than m +n — 1 comparisons. 


Sometimes two sorted lists of length m and n can be merged using far fewer than m+ 
n — | comparisons. For instance, when m = 1, a binary search procedure can be applied to put 
the one element in the first list into the second list. This requires only [logn] comparisons, 
which is much smaller than m + n — 1 =n, for m = 1. On the other hand, for some values of m 
and n, Lemma 1 gives the best possible bound. That is, there are lists with m and n elements 
that cannot be merged using fewer than m + n — 1 comparisons. (See Exercise 47.) 

We can now analyze the complexity of the merge sort. Instead of studying the general 
problem, we will assume that n, the number of elements in the list, is a power of 2, say 2”. This 
will make the analysis less complicated, but when this is not the case, various modifications can 
be applied that will yield the same estimate. 

At the first stage of the splitting procedure, the list is split into two sublists, of 2””-! elements 
each, at level 1 of the tree generated by the splitting. This process continues, splitting the two 
sublists with 2”-! elements into four sublists of 2”? elements each at level 2, and so on. In 
general, there are 2k! lists at level k — 1, each with 2”-*+! elements. These lists at level k — 1 
are split into 2* lists at level k, each with 2”~* elements. At the end of this process, we have 2” 
lists each with one element at level m. 

We start merging by combining pairs of the 2” lists of one element into 2’""! lists, at level 
m — 1, each with two elements. To do this, 2”! pairs of lists with one element each are merged. 
The merger of each pair requires exactly one comparison. 

The procedure continues, so that at level k (k = m, m — 1,m — 2, ..., 3, 2, 1), 2* lists each 
with 2”-* elements are merged into 2‘! lists, each with 2””-*t! elements, at level k — 1. To do 
this a total of 24! mergers of two lists, each with 2"-k elements, are needed. But, by Lemma 1, 
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each of these mergers can be carried out using at most 2”74 + 2”-* — 1 = 2"-*+! — 1 compar- 
isons. Hence, going from level k to k — 1 can be accomplished using at most 2%! (2"-4+1 — 1) 
comparisons. 

Summing all these estimates shows that the number of comparisons required for the merge 
sort is at most 


m m m 


2 IEL Oa _ 1) = > qm 3 gk! = m” — o” _ 1) = nlogn =ñ E 1, 
k=1 k k 


=] =1 


because m = logn andn = 2”. (We evaluated )7"_ , 2” by noting that it is the sum of m identical 
terms, each equal to 2”. We evaluated )"”"_ | 2‘~' using the formula for the sum of the terms of 
a geometric progression from Theorem | of Section 2.4.) 

Theorem 1 summarizes what we have discovered about the worst-case complexity of the 


merge sort algorithm. 


The number of comparisons needed to merge sort a list with n elements is O(n log n). 


In Chapter 11 we will show that the fastest comparison-based sorting algorithm have 
O(n log n) time complexity. (A comparison-based sorting algorithm has the comparison of two 
elements as its basic operation.) Theorem 1 tells us that the merge sort achieves this best pos- 
sible big-O estimate for the complexity of a sorting algorithm. We describe another efficient 
algorithm, the quick sort, in the preamble to Exercise 50. 


. Trace Algorithm 1 when it is given n = 5 as input. That 
is, show all steps used by Algorithm 1 to find 5!, as is 
done in Example | to find 4!. 

. Trace Algorithm 1 when it is given n = 6 as input. That 
is, show all steps used by Algorithm 1 to find 6!, as is 
done in Example | to find 4!. 

. Trace Algorithm 3 when it finds gcd(8, 13). That is, show 
all the steps used by Algorithm 3 to find gcd(8, 13). 

. Trace Algorithm 3 when it finds gcd(12, 17). That 
is, show all the steps used by Algorithm 3 to find 
gcd(12, 17). 

. Trace Algorithm 4 when it is given m = 5, n = 11, and 
b = 3 as input. That is, show all the steps Algorithm 4 
uses to find 3!!mod 5. 

. Trace Algorithm 4 when it is given m = 7, n = 10, and 
b =2 as input. That is, show all the steps Algorithm 4 
uses to find 2'!°mod 7. 

. Give a recursive algorithm for computing nx whenever n 
is a positive integer and x is an integer, using just addi- 
tion. 

. Give a recursive algorithm for finding the sum of the 
first n positive integers. 

. Give a recursive algorithm for finding the sum of the 
first n odd positive integers. 


11. 


12. 


13. 


14. 


15. 


16. 


. Give a recursive algorithm for finding the maximum of a 


finite set of integers, making use of the fact that the max- 
imum of 7 integers is the larger of the last integer in the 
list and the maximum of the first n — 1 integers in the list. 


Give a recursive algorithm for finding the minimum of a 
finite set of integers, making use of the fact that the min- 
imum of n integers is the smaller of the last integer in the 
list and the minimum of the first n — 1 integers in the list. 


Devise a recursive algorithm for finding x” mod m when- 
ever n, x, and m are positive integers based on the fact that 
x" mod m = (x7! mod m - x mod m) mod m. 


Give a recursive algorithm for finding n! mod m when- 
ever n and m are positive integers. 


Give a recursive algorithm for finding a mode of a list of 
integers. (A mode is an element in the list that occurs at 
least as often as every other element.) 


Devise a recursive algorithm for computing the greatest 
common divisor of two nonnegative integers a and b with 
a < busing the fact that gcd(a, b) = gcd(a, b — a). 


Prove that the recursive algorithm for finding the sum of 
the first positive integers you found in Exercise 8 is 
correct. 
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17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


*26. 


* 27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


37. 


Describe a recursive algorithm for multiplying two non- 
negative integers x and y based on the fact that xy = 2(x - 
(y/2)) when y is even and xy = 2(x- |y/2]) +x when y 
is odd, together with the initial condition xy = 0 when 
y=0. 

Prove that Algorithm 1 for computing n! when n is a non- 
negative integer is correct. 

Prove that Algorithm 3 for computing gcd(a, b) when a 
and b are positive integers with a < b is correct. 

Prove that the algorithm you devised in Exercise 17 is 
correct. 

Prove that the recursive algorithm that you found in Ex- 
ercise 7 is correct. 

Prove that the recursive algorithm that you found in Ex- 
ercise 10 is correct. 

Devise a recursive algorithm for computing n? where n 
is a nonnegative integer, using the fact that (n+ 1)? = 
n? + 2n + 1. Then prove that this algorithm is correct. 
Devise a recursive algorithm to find a", where a is a real 
number and n is a positive integer. [Hint: Use the equal- 
ity a2" = (a")?.] 

How does the number of multiplications used by the al- 
gorithm in Exercise 24 compare to the number of multi- 
plications used by Algorithm 2 to evaluate a°”? 

Use the algorithm in Exercise 24 to devise an algo- 
rithm for evaluating a” when n is a nonnegative integer. 
[Hint: Use the binary expansion of n.] 

How does the number of multiplications used by the al- 
gorithm in Exercise 26 compare to the number of multi- 
plications used by Algorithm 2 to evaluate a”? 

How many additions are used by the recursive and itera- 
tive algorithms given in Algorithms 7 and 8, respectively, 
to find the Fibonacci number f}? 


Devise a recursive algorithm to find the nth term of the 
sequence defined by dy = 1,a, = 2, anda, =a,_ 
forn = 2,3,4,.... 


Devise an iterative algorithm to find the nth term of the 
sequence defined in Exercise 29. 

Is the recursive or the iterative algorithm for finding the 
sequence in Exercise 29 more efficient? 

Devise a recursive algorithm to find the nth term of the 
sequence defined by ay) = 1, a, = 2, a = 3, and a, = 
an-ı + Ay_2 + 4y_3, for n = 3,4,5,.... 

Devise an iterative algorithm to find the nth term of the 
sequence defined in Exercise 32. 

Is the recursive or the iterative algorithm for finding the 
sequence in Exercise 32 more efficient? 

Give iterative and recursive algorithms for finding the nth 
term of the sequence defined by ag = 1, a, = 3, a, = 5, 
and a, = d,_1° a, . ae Which is more efficient? 
Give a recursive algorithm to find the number of parti- 
tions of a positive integer based on the recursive defini- 
tion given in Exercise 49 in Section 5.3. 

Give a recursive algorithm for finding the reversal of a bit 
string. (See the definition of the reversal of a bit string in 
the preamble of Exercise 36 in Section 5.3.) 


*Gn-2 Links) 


38. Give a recursive algorithm for finding the string w’, the 
concatenation of i copies of w, when w is a bit string. 


39. Prove that the recursive algorithm for finding the reversal 
of a bit string that you gave in Exercise 37 is correct. 


40. Prove that the recursive algorithm for finding the concate- 
nation of i copies of a bit string that you gave in Exercise 
38 is correct. 


*41. Give a recursive algorithm for tiling a 2” x 2” checker- 
board with one square missing using right triominoes. 


42. Give a recursive algorithm for triangulating a simple 
polygon with n sides, using Lemma 1 in Section 5.2. 


43. Give a recursive algorithm for computing values of the 
Ackermann function. [Hint: See the preamble to Exer- 
cise 48 in Section 5.3.] 


44. Use a merge sort to sort 4, 3, 2, 5, 1, 8, 7, 6 into increasing 
order. Show all the steps used by the algorithm. 


45. Use a merge sort to sort b, d, a, f, g, h, z, p, o, k into al- 
phabetic order. Show all the steps used by the algorithm. 


46. How many comparisons are required to merge these pairs 
of lists using Algorithm 10? 
a) 1,3, 5,7, 9; 2, 4, 6, 8, 10 
b) 1,2,3,4, 5; 6, 7, 8,9, 10 
c) 1,5, 6,7, 8; 2, 3, 4, 9, 10 

47. Show that for all positive integers m and n there are sorted 
lists with m elements and n elements, respectively, such 
that Algorithm 10 uses m + n — 1 comparisons to merge 
them into one sorted list. 


* 48. What is the least number of comparisons needed to merge 
any two lists in increasing order into one list in increasing 
order when the number of elements in the two lists are 
a) 1,4? b) 2, 4? c) 3, 4? d) 4, 4? 


*49, Prove that the merge sort algorithm is correct. 


The quick sort is an efficient algorithm. To sort 
4, dy, ...,a,, this algorithm begins by taking the first ele- 
ment a, and forming two sublists, the first containing those 
elements that are less than q4, in the order they arise, and the 
second containing those elements greater than a}, in the or- 
der they arise. Then a, is put at the end of the first sublist. 
This procedure is repeated recursively for each sublist, until 
all sublists contain one item. The ordered list of n items is 
obtained by combining the sublists of one item in the order 
they occur. 


50. Sort 3, 5, 7, 8, 1, 9, 2, 4, 6 using the quick sort. 

51. Let ay, a, ..., a, bea list of n distinct real numbers. How 
many comparisons are needed to form two sublists from 
this list, the first containing elements less than a, and the 
second containing elements greater than a,? 

52. Describe the quick sort algorithm using pseudocode. 

53. What is the largest number of comparisons needed to or- 
der a list of four elements using the quick sort algorithm? 

54. What is the least number of comparisons needed to order 
a list of four elements using the quick sort algorithm? 


55. Determine the worst-case complexity of the quick sort 
algorithm in terms of the number of comparisons used. 
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Program Correctness 
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Links > 


5.5.1 Introduction 


Suppose that we have designed an algorithm to solve a problem and have written a program 
to implement it. How can we be sure that the program always produces the correct answer? 
After all the bugs have been removed so that the syntax is correct, we can test the program with 
sample input. It is not correct if an incorrect result is produced for any sample input. But even if 
the program gives the correct answer for all sample input, it may not always produce the correct 
answer (unless all possible input has been tested). We need a proof to show that the program 
always gives the correct output. 

Program verification, the proof of correctness of programs, uses the rules of inference and 
proof techniques described in this chapter, including mathematical induction. Because an incor- 
rect program can lead to disastrous results, a large amount of methodology has been constructed 
for verifying programs. Efforts have been devoted to automating program verification so that it 
can be carried out using a computer. However, only limited progress has been made toward this 
goal. Indeed, some mathematicians and theoretical computer scientists argue that it will never 
be realistic to mechanize the proof of correctness of complex programs. 

Some of the concepts and methods used to prove that programs are correct will be intro- 
duced in this section. Many different methods have been devised for proving that programs are 
correct. We will discuss a widely used method for program verification introduced by Tony 
Hoare in this section; several other methods are also commonly used. Furthermore, we will not 
develop a complete methodology for program verification in this book. This section is meant to 
be a brief introduction to the area of program verification, which ties together the rules of logic, 
proof techniques, and the concept of an algorithm. 


5.5.2 Program Verification 


A program is said to be correct if it produces the correct output for every possible input. A 
proof that a program is correct consists of two parts. The first part shows that the correct answer 
is obtained if the program terminates. This part of the proof establishes the partial correctness 
of the program. The second part of the proof shows that the program always terminates. 

To specify what it means for a program to produce the correct output, two propositions 
are used. The first is the initial assertion, which gives the properties that the input values must 
have. The second is the final assertion, which gives the properties that the output of the program 
should have, if the program did what was intended. The appropriate initial and final assertions 
must be provided when a program is checked. 


A program, or program segment, S is said to be partially correct with respect to the 
initial assertion p and the final assertion q if whenever p is true for the input values 
of S and S terminates, then q is true for the output values of S. The notation p{S}q indi- 
cates that the program, or program segment, S is partially correct with respect to the initial 
assertion p and the final assertion q. 


Note: The notation p{S}g is known as a Hoare triple. Tony Hoare introduced the concept of 
partial correctness. 

Note that the notion of partial correctness has nothing to do with whether a program termi- 
nates; it focuses only on whether the program does what it is expected to do if it terminates. 

A simple example illustrates the concepts of initial and final assertions. 
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Extra 


Show that the program segment 


y= 2 
Zi=xt+y 


Examples » is correct with respect to the initial assertion p: x = 1 and the final assertion q: z = 3. 


Solution: Suppose that p is true, so that x = 1 as the program begins. Then y is assigned the 
value 2, and z is assigned the sum of the values of x and y, which is 3. Hence, S is correct with 
respect to the initial assertion p and the final assertion q. Thus, p{S}q is true. < 


5.5.3 Rules of Inference 


A useful rule of inference proves that a program is correct by splitting the program into a se- 
quence of subprograms and then showing that each subprogram is correct. 

Suppose that the program S is split into subprograms S} and $,. Write S = S,; S, to indicate 
that S is made up of S; followed by S,. Suppose that the correctness of S} with respect to the 
initial assertion p and final assertion q, and the correctness of S, with respect to the initial 
assertion q and the final assertion r, have been established. It follows that if p is true and S, is 
executed and terminates, then q is true; and if q is true, and S, executes and terminates, then r 
is true. Thus, if p is true and S = S; S, is executed and terminates, then r is true. This rule of 
inference, called the composition rule, can be stated as 


PIS\}q 
q{ Sy }r 


z. {S13 S» }r. 


This rule of inference will be used later in this section. 

Next, some rules of inference for program segments involving conditional statements and 
loops will be given. Because programs can be split into segments for proofs of correctness, this 
will let us verify many different programs. 


5.5.4 Conditional Statements 


First, rules of inference for conditional statements will be given. Suppose that a program 
segment has the form 


if condition then 
S 


where S is a block of statements. Then S is executed if condition is true, and it is not executed 
when condition is false. To verify that this segment is correct with respect to the initial asser- 
tion p and final assertion q, two things must be done. First, it must be shown that when p is true 
and condition is also true, then q is true after S terminates. Second, it must be shown that when 
p is true and condition is false, then q is true (because in this case S does not execute). 

This leads to the following rule of inference: 


(p A condition){S}q 
(p A acondition) > q 


.. p {if condition then S}q. 


EXAMPLE 2 


Courtesy of Tony Hoare 
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Example 2 illustrates how this rule of inference is used. 


Verify that the program segment 


if x > y then 
yi=x 


is correct with respect to the initial assertion T and the final assertion y > x. 


Solution: When the initial assertion is true and x > y, the assignment y := x is carried out. 
Hence, the final assertion, which asserts that y > x, is true in this case. Moreover, when the 
initial assertion is true and x > y is false, so that x < y, the final assertion is again true. Hence, 
using the rule of inference for program segments of this type, this program is correct with respect 
to the given initial and final assertions. < 


Similarly, suppose that a program has a statement of the form 


if condition then 
Si 

else 
S2 


If condition is true, then S, executes; if condition is false, then S, executes. To verify that this 
program segment is correct with respect to the initial assertion p and the final assertion g, two 
things must be done. First, it must be shown that when p is true and condition is true, then q 
is true after S} terminates. Second, it must be shown that when p is true and condition is false, 
then q is true after S, terminates. This leads to the following rule of inference: 


(p A condition){S, }q 
(p A acondition) {Sy }q 


.. p{if condition then S, else S,}q. 


C. ANTHONY R. HOARE (BORN 1934) Tony Hoare was born in Colombo, Ceylon (now known as Sri 
Lanka), where his father was a civil servant of the British Empire and his mother’s father owned a plantation. 
He spent his early childhood in Ceylon, moving to England in 1945. Hoare studied philosophy, together with 
the classics, at the University of Oxford, where he became interested in computing as a result of his fascination 
with the power of mathematical logic and the certainty of mathematical truth. He received his bachelors degree 
from Oxford in 1956. 

Hoare learned Russian during his service in the Royal Navy, and later studied the computer transla- 
tion of natural languages at Moscow State University. He returned to England in 1960, taking a job at a 
small computer manufacturer, where he wrote a compiler for the programming language Algol. In 1968, 
he became Professor of Computing Science at the Queen’s University, Belfast; in 1977, he moved to the 


University of Oxford as Professor of Computing; he is now Professor Emeritus. He is a Fellow of the Royal Society and also holds a 
position at Microsoft Research in Cambridge. 

Hoare has made many contributions to the theory of programming languages and to programming methodology. He was first 
to define a programming language based on how programs could be proved correct with respect to their specifications. Hoare also 
invented quick sort, one of the most commonly used sorting algorithms (see the preamble to Exercise 50 in Section 5.4). He received 
the ACM Turing Award in 1980 and in 2000 he was knighted for services to education and computer science. Hoare is a noted writer 
in the technical and social aspects of computer science. 
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EXAMPLE 3 


Links 


» 


EXAMPLE 4 


Extra 
Examples 


» 


Example 3 illustrates how this rule of inference is used. 


Verify that the program segment 


if x < 0 then 
abs := —x 
else 
abs := x 


is correct with respect to the initial assertion T and the final assertion abs = |x|. 


Solution: Two things must be demonstrated. First, it must be shown that if the initial assertion 
is true and x < 0, then abs = |x|. This is correct, because when x < 0 the assignment statement 
abs := —x sets abs = —x, which is |x| by definition when x < 0. Second, it must be shown that 
if the initial assertion is true and x < 0 is false, so that x > 0, then abs = |x|. This is also cor- 
rect, because in this case the program uses the assignment statement abs := x, and x is |x| by 
definition when x > 0, so abs := x. Hence, using the rule of inference for program segments of 
this type, this segment is correct with respect to the given initial and final assertions. < 


5.5.5 Loop Invariants 


Next, proofs of correctness of while loops will be described. To develop a rule of inference for 
program segments of the type 


while condition 
S 


note that S is repeatedly executed until condition becomes false. An assertion that remains true 
each time S is executed must be chosen. Such an assertion is called a loop invariant. In other 
words, p is a loop invariant if (p A condition) {S}p is true. 

Suppose that p is a loop invariant. It follows that if p is true before the program segment is 
executed, p and -condition are true after termination, if it occurs. This rule of inference is 


(p A condition) {S}p 


.“. p{ while condition S}(- condition A p). 


The use of a loop invariant is illustrated in Example 4. 


A loop invariant is needed to verify that the program segment 


i:=1 
factorial := 1 
while i < n 
i:=itl 
factorial := factorial «i 


terminates with factorial = n! when n is a positive integer. 


EXAMPLE 5 
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Let p be the assertion “factorial = i! and i < n.” We first prove that p is a loop invariant. 
Suppose that, at the beginning of one execution of the while loop, p is true and the condition of 
the while loop holds; in other words, assume that factorial = i! and that i < n. The new values 
inew and factorial,,,, of i and factorial are i,.., = i+ 1 and factorial,,,, = factorial -(i+1)= 
(i+ 1)! = iew! Because i < n, we also have i, =i+1 <n. Thus, p is true at the end of the 
execution of the loop. This shows that p is a loop invariant. 

Now we consider the program segment. Just before entering the loop, i= 1 <n and 
factorial = 1 = 1! =i! both hold, so p is true. Because p is a loop invariant, the rule of in- 
ference just introduced implied that if the while loop terminates, it terminates with p true and 
with i < n false. In this case, at the end, factorial = i! and i < n are true, but i < n is false; in 
other words, i = n and factorial = i! = n!, as desired. 

Finally, we need to check that the while loop actually terminates. At the beginning of the 
program i is assigned the value 1, so after n — 1 traversals of the loop, the new value of i will 
be n, and the loop terminates at that point. < 


A final example will be given to show how the various rules of inference can be used to 
verify the correctness of a longer program. 


We will outline how to verify the correctness of the program S for computing the product of 
two integers. 


procedure multiply(m, n: integers) 


if n < 0 then a := -n 
1 \ elsea:=n 


k:=0 
aa 


while k < a 
S34 x:=x+m 
k:=k+1 


if n < 0 then product := —x 
4 \ else product := x 


return product 
{product equals mn} 


The goal is to prove that after S is executed, product has the value mn. The proof of correctness 
can be carried out by splitting S into four segments, with S = S1; S2; S3; S4, as shown in the 
listing of S. The rule of composition can be used to build the correctness proof. Here is how the 
argument proceeds. The details will be left as an exercise for the reader. 

Let p be the initial assertion “m and n are integers.” Then, it can be shown that p{S, }q is true, 
when q is the proposition p A (a = |n|). Next, let r be the proposition q A (k = 0) A (x = 0). It is 
easily verified that q {S;, }r is true. It can be shown that “x = mk and k < a” is an invariant for the 
loop in S}. Furthermore, it is easy to see that the loop terminates after a iterations, with k = a, 
so x = ma at this point. Because r implies that x = m- O and O < a, the loop invariant is true 
before the loop is entered. Because the loop terminates with k = a, it follows that r{ S; }s is true, 
where s is the proposition “x = ma and a = |n|.” Finally, it can be shown that S, is correct with 
respect to the initial assertion s and final assertion t, where f is the proposition “product = mn.” 
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Putting all this together, because p{S,}q, q{S,}r, r{S3}s, and s{S,}t are all true, it fol- 
lows from the rule of composition that p{S}t is true. Furthermore, because all four segments 


terminate, S does terminate. This verifies the correctness of the program. < 
Exercises 
1. Prove that the program segment if x < 0 then 
y:=1 y := —2|x|/x 
zisxty else if os an 
= 2|x|/x 
is correct with respect to the initial assertion x = 0 and a if x = 0 then 
the final assertion z = 1. y:=2 
2. Verify that the program segment is correct with respect to the initial assertion T and the 
fr<öthenx:=0 final assertion y = 2. 


7. Use a loop invariant to prove that the following program 
segment for computing the nth power, where n is a posi- 
tive integer, of a real number x is correct. 


is correct with respect to the initial assertion T and the 
final assertion x > 0. 


3. Verify that the program segment power := 1 
KH 2 e 
z:=x+y while i < n 
if y > 0 then power := power * x 
z:=z+1 i:=i+1 
else *8. Prove that the iterative program for finding f, given in 
z:=0 Section 5.4 is correct. 
is correct with respect to the initial assertion y = 3 and 9. Provide all the details in the proof of correctness given in 
the final assertion z = 6. Example 5. 
4. Verify that the program segment 10. Suppose that both the conditional statement pọ > p, and 
if x < y then the program assertion p, {S}q are true. Show that py {S}q 
Hina x also must be true. 
else 11. Suppose that both the program assertion p{S}q, and the 
min := y conditional statement gy — q; are true. Show that p{S}q; 


F i i a ; also must be true. 
is correct with respect to the initial assertion T and the 


final assertion (x < y A min = x) V (x > y A min = y). 12. This program computes quotients and remainders. 


; : ; ; ; r:=a 
*5, Devise a rule of inference for verification of partial cor- asi 
rectness of statements of the form oe 
if conditi h while r > d 
if condition 1 then Peed 
Si, n q:=q+1 
else if condition 2 then . fe : : das 
5 Verify that it is partially correct with respect to the ini- 
@ : tial assertion “a and d are positive integers” and the final 
else assertion “q and r are integers such that a = dq + r and 


S O<r<d” 
É 13. Use a loop invariant to verify that the Euclidean algo- 


WIETE Spaas} Sp ATE Blocks rithm (Algorithm 1 in Section 4.3) is partially correct 


6. Use the rule of inference developed in Exercise 5 to ver- with respect to the initial assertion “a and b are positive 
ify that the program integers” and the final assertion “x = gcd(a, b).” 
Key Terms and Results 
TERMS arithmetic progression: a sequence of the form a, a +d, 


sequence: a function with domain that is a subset of the set of at 2d.: Whereaahd d are teal numbers 


integers 
the principle of mathematical induction: The statement 


geometric progression: a sequence of the form a, ar, ar, ..., Vn P(n) is true if P(1) is true and Vk[P(k) > P(k + 1)] is 
where a and r are real numbers true. 


basis step: the proof of P(1) in a proof by mathematical in- 
duction of VnP(n) 


inductive step: the proof of P(k) > P(k +1) for all posi- 
tive integers k in a proof by mathematical induction of 
VnP(n) 


strong induction: The statement VnP(n) is true if P(1) is true 
and Vk[(P(1) A + A P(k)) > P(k + 1)] is true. 


well-ordering property: Every nonempty set of nonnegative 
integers has a least element. 


recursive definition of a function: a definition of a function 
that specifies an initial set of values and a rule for obtaining 
values of this function at integers from its values at smaller 
integers 


recursive definition of a set: a definition of a set that specifies 
an initial set of elements in the set and a rule for obtaining 
other elements from those in the set 


Review Questions 


1. a) Can you use the principle of mathematical induction 
to find a formula for the sum of the first n terms of a 
sequence? 

b) Can you use the principle of mathematical induction 
to determine whether a given formula for the sum of 
the first n terms of a sequence is correct? 

c) Find a formula for the sum of the first n even positive 
integers, and prove it using mathematical induction. 

2. a) For which positive integers nis 11n +17 < 2”? 

b) Prove the conjecture you made in part (a) using math- 
ematical induction. 


3. a) Which amounts of postage can be formed using only 
5-cent and 9-cent stamps? 
b) Prove the conjecture you made using mathematical in- 
duction. 
c) Prove the conjecture you made using strong induction. 
d) Find a proof of your conjecture different from the ones 
you gave in (b) and (c). 
4. Give two different examples of proofs that use strong in- 
duction. 
5. a) State the well-ordering property for the set of positive 
integers. 
b) Use this property to show that every positive inte- 
ger greater than one can be written as the product of 
primes. 


6. a) Explain why a function f from the set of positive in- 
tegers to the set of real numbers is well defined if it is 
defined recursively by specifying f(1) and a rule for 
finding f (n) from f(n — 1). 

b) Provide a recursive definition of the function f(n) = 
(n+ 1)!. 

7. a) Give arecursive definition of the Fibonacci numbers. 

b) Show that f, > a”? whenever n > 3, where f, is the 

nth term of the Fibonacci sequence and a = (1 + 


V5)/2. 
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structural induction: a technique for proving results about re- 
cursively defined sets 

recursive algorithm: an algorithm that proceeds by reducing 
a problem to the same problem with smaller input 

merge sort: a sorting algorithm that sorts a list by splitting it 
in two, sorting each of the two resulting lists, and merging 
the results into a sorted list 

iteration: a procedure based on the repeated use of operations 
in a loop 

program correctness: verification that a procedure always 
produces the correct result 

loop invariant: a property that remains true during every 
traversal of a loop 

initial assertion: the statement specifying the properties of the 
input values of a program 

final assertion: the statement specifying the properties the 
output values should have if the program worked correctly 


8. a) Explain why a sequence a,, is well defined if it is de- 

fined recursively by specifying a, and a, and a rule 

for finding a,, from ay, az, ..., d,_, for n = 3,4,5,.... 

b) Find the value of a, ifa, = 1,a, = 2, and a„ =a,_,+ 
d,-2 +++ + ay, forn = 3,4,5,.... 


9. Give two examples of how well-formed formulae are 
defined recursively for different sets of elements and 
operators. 


10. a) Give a recursive definition of the length of a string. 
b) Use the recursive definition from part (a) and struc- 
tural induction to prove that /(xy) = I(x) + l). 


11. a) What is a recursive algorithm? 
b) Describe a recursive algorithm for computing the sum 
of n numbers in a sequence. 


12. Describe a recursive algorithm for computing the greatest 
common divisor of two positive integers. 


13. a) Describe the merge sort algorithm. 
b) Use the merge sort algorithm to put the list 4, 10, 1, 
5, 3, 8, 7, 2, 6, 9 in increasing order. 
c) Give a big-O estimate for the number of comparisons 
used by the merge sort. 


14. a) Does testing a computer program to see whether it 
produces the correct output for certain input values 
verify that the program always produces the correct 
output? 

b) Does showing that a computer program is partially 
correct with respect to an initial assertion and a final 
assertion verify that the program always produces the 
correct output? If not, what else is needed? 


15. What techniques can you use to show that a long com- 
puter program is partially correct with respect to an initial 
assertion and a final assertion? 


16. What is a loop invariant? How is a loop invariant used? 
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Supplementary Exercises 


1. 


14. 


15. 


16. 


Use toate mae induction to show that Z += a4 5 Ls ar 


2 
ae ee L whenever n is a positive integer 


Use endl induction to show that 1° + 33 + 53 + 
+ (2n+ 1)3 = (n + 1)?(2n? + 4n + 1) whenever n is a 
nica integer. 


Use mathematical induction to show that 1 -29+2-2! + 
3-22 +- +n- 27! =(n—1)-2"+4+1 whenever n is a 
positive integer. 

Use mathematical induction to show that 


1 Lo 1 > n 


T335 t *@i-D@n+ n+l 
whenever n is a positive integer. 
. Show that 
1 + 1 Pisi 1 m n 
1-4 4-7 (3n—2)3n+1) 3n+1 


whenever n is a positive integer. 


. Use mathematical induction to show that 2” > n? +n 


whenever n is an integer greater than 4. 


. Use mathematical induction to show that 2” > n? when- 


ever n is an integer greater than 9. 


. Find an integer N such that 2” > n* whenever n is an inte- 


ger greater than N. Prove that your result is correct using 
mathematical induction. 


. Use mathematical induction to prove that a — b is a factor 


of a” — b” whenever n is a positive integer. 


. Use mathematical induction to prove that 9 divides n? + 


(n+ 1)? + (n + 2)? whenever n is a nonnegative integer. 


. Use mathematical induction to prove that 43 divides 
6 


n+l 472-1 for every positive integer n. 


. Use mathematical induction to prove that 64 divides 


37"? + 56n + 55 for every positive integer n. 


. Use mathematical induction to prove this formula for the 


sum of the terms of an arithmetic progression. 


a+(atd)+-++(at+nd) = (n+ 1)(2a+nd)/2 


Suppose that a= b (mod m) for j= 1,2,...,n. Use 


mathematical induction to prove that 


a) Ya vb 


i a 


. (mod m). 


b) ii q = i b (mod m). 
j=1 j=1 
Show that if n is a positive integer, then 


k+4 


—— _ Bnat) 
“kk + I(k +2) 


~ 2(n + 1)(n + 2) 


For which positive integers n is n +6 < (n? — 8n)/16? 
Prove your answer using mathematical induction. 


17. 


18. 


*19. 


* 20. 


*21. 


(Requires calculus) Suppose that f(x) = e* and g(x) = 
xe*. Use mathematical induction together with the prod- 
uct rule and the fact that f'(x) = e* to prove that g(x) = 
(x + n)e* whenever n is a positive integer. 


(Requires calculus) Suppose that f(x) = e* and g(x) = 
e“, where c is a constant. Use mathematical induction 
together with the chain rule and the fact that f’(x) = e* to 
prove that g = c"e whenever n is a positive integer. 


Formulate a conjecture about which Fibonacci numbers 
are even, and use a form of mathematical induction to 
prove your conjecture. 


Determine which Fibonacci numbers are divisible by 3. 
Use a form of mathematical induction to prove your con- 
jecture. 


Prove that ffa + fkyifn41 = fagga for all nonnegative in- 
tegers n and k, where f; denotes the ith Fibonacci number. 


Recall from Example 15 of Section 2.4 that the sequence of 
Lucas numbers is defined by /, = 2, 4 = 1, and /, =1,_) + 
l,—2 forn = 2, 3,4, . 


22. 


23. 


*24. 


25. 


*26. 


27. 


28. 


29. 


Show that f, + foun = l„„ı whenever n is a positive inte- 
ger, where f; and l; are ‘ie ith Fibonacci number and ith 
Lucas number, respectively. 


Show that + p +- +Ê =1,1,,; +2 whenever n is a 
nonnegative integer and /; is the ith Lucas number. 


Use mathematical induction to show that the product of 
any consecutive positive integers is divisible by n!. 
[Hint: Use the identity m(m+1)+--(m+n-—1)/n! = 
(m— 1)m(m + 1) ++ (m +n — 2)/n!+m(m+ 1) (m+ 
n—2)/(n—1)!.] 

Use mathematical induction to show that (cosx+ 
isinx)" = cosnx +isinnx whenever n is a positive in- 
teger. (Here i is the square root of —1.) [Hint: Use 
the identities cos(a + b) =cosacosb—sinasinb and 
sin(a + b) = sina cos b + cos asin b.] 


Use mathematical induction to show that 2i 
cos[(n + 1)x/2] sin(nx/2)/ sin(x/2) whenever n is a pos- 
itive integer and sin(x/2) # 0. 


cos jx = 


Use mathematical induction to prove that X; , Pus 
n?2"+1 — n2"+2 4.3 . 2"+1 _ 6 for every positive integer n. 


calculus) Suppose that the 
is recursively defined by x, 


(Requires 


HypXos.< og Hye fos 


Xn+1 = VXn +6. 

a) Use mathematical induction to show that x, < x, < 
<x, < +++, that is, the sequence {x,} is monotoni- 
cally increasing. 

b) Use mathematical induction to prove that x, < 3 for 
WH, Dex 

c) Show that lim 

Show if 7 is a positive integer with n > 2, then 


sequence 
= 0 and 


x= 3, 


NO "nN 


n 


5 1 — @—1)Gn+2) 
=) j-1 4n(n+1) ` 


30. 


*31, 


32. 


33. 


34. 


35. 


* 36. 


37. 


* 38. 


39. 


* 40. 


Use mathematical induction to prove Theorem | in Sec- 
tion 4.2, that is, show if b is an integer, where b > 1, and 
n is a positive integer, then n can be expressed uniquely 
in the form n = a,b‘ + a,_,b!! + ++ +a,b + do. 

A lattice point in the plane is a point (x, y) where both x 
and y are integers. Use mathematical induction to show 
that at least n + 1 straight lines are needed to ensure that 
every lattice point (x, y) with x > 0, y >0,andx+y<n 
lies on one of these lines. 

(Requires calculus) Use mathematical induction and the 
product rule to show that if n is a positive integer and 
fO, fo), ....f,@), are all differentiable functions, then 


(FOO) fy 

AO fy 
AO FO fe) 
Ai@ RO FO) 


(Requires material in Section 2.6) Suppose that B = 
MAM”™!, where A and B are nxn matrices and M is 
invertible. Show that B* = MA‘M7! for all positive in- 
tegers k. (Consult both the text of Section 2.6 and the 
preamble to Exercise 18 of Section 2.6.) 


Use mathematical induction to show that if you draw lines 
in the plane you only need two colors to color the regions 
formed so that no two regions that have an edge in com- 
mon have a common color. 

Show that n! can be represented as the sum of n of its 
distinct positive divisors whenever n > 3. [Hint: Use in- 
ductive loading. First try to prove this result using math- 
ematical induction. By examining where your proof fails, 
find a stronger statement that you can easily prove using 
mathematical induction.] 

Use mathematical induction to prove that if x,, x, ..., Xn 
are positive real numbers with n > 2, then 


2s 

Xi 7) Xn 

rea e 

Xyt— jot Jae (xp +— te 
X2 X3 Ke X 


Use mathematical induction to prove that if n people 
stand in a line, where n is a positive integer, and if the 
first person in the line is a woman and the last person in 
line is a man, then somewhere in the line there is a woman 
directly in front of a man. 

Suppose that for every pair of cities in a country there is a 
direct one-way road connecting them in one direction or 
the other. Use mathematical induction to show that there 
is a city that can be reached from every other city either 
directly or via exactly one other city. 

Use mathematical induction to show that when n circles 
divide the plane into regions, these regions can be col- 
ored with two different colors such that no regions with a 
common boundary are colored the same. 

Suppose that among a group of cars on a circular track 
there is enough fuel for one car to complete a lap. Use 
mathematical induction to show that there is a car in the 


41. 


42. 


*43. 


44. 
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group that can complete a lap by obtaining gas from other 
cars as it travels around the track. 


Show that if n is a positive integer, then 


Zo- 1) (È u) = n(n + 1)/2. 
j=1 k=j 


Use mathematical induction to show that if a, b, and c 
are the lengths of the sides of a right triangle, where c 
is the length of the hypotenuse, then a” + b” < c” for all 
integers n with n > 3. 


Use mathematical induction to show that if n is a posi- 
tive integers, the sequence 2 mod n, 2? mod n, 27 mod n, 


2?” mod n, ... is eventually constant (that is, all terms 
after a finite number of terms are all the same). 


A unit or Egyptian fraction is a fraction of the form 

1/n, where n is a positive integer. In this exercise, we 

will use strong induction to show that a greedy algorithm 

can be used to express every rational number p/q with 

0 < p/q < 1 as the sum of distinct unit fractions. At each 

step of the algorithm, we find the smallest positive integer 

n such that 1/n can be added to the sum without exceed- 

ing p/q. For example, to express 5/7 we first start the 

sum with 1/2. Because 5/7 — 1/2 = 3/14 we add 1/5 to 
the sum because 5 is the smallest positive integer k such 
that 1/k < 3/14. Because 3/14 — 1/5 = 1/70, the algo- 

rithm terminates, showing that 5/7 = 1/2 + 1/5 + 1/70. 

Let T(p) be the statement that this algorithm terminates 

for all rational numbers p/q with 0 < p/q < 1. We will 

prove that the algorithm always terminates by showing 

that T(p) holds for all positive integers p. 

a) Show that the basis step T(1) holds. 

b) Suppose that T(k) holds for positive integers k with 
k < p. That is, assume that the algorithm terminates 
for all rational numbers k/r, where 1 < k < p. Show 
that if we start with p/q and the fraction 1/n is se- 
lected in the first step of the algorithm, then p/q = 
p'/q' +1/n, where p' =np—q and q' = nq. After 
considering the case where p/q = 1/n, use the induc- 
tive hypothesis to show that the greedy algorithm ter- 
minates when it begins with p’/q’ and complete the 
inductive step. 


The McCarthy 91 function (defined by John McCarthy, one 
of the founders of artificial intelligence) is defined using the 


rule 


ifn > 100 


n—10 
M(n) = com ifn < 100 


for all positive integers n. 


45. 


**46. 


By successively using the defining rule for M(n), find 

a) M(102). b) M(101). c) M(99). 

d) M(97). e) M(87). f) M(76). 

Show that the function M(n) is a well-defined function 
from the set of positive integers to the set of positive inte- 
gers. [Hint: Prove that M(n) = 91 for all positive integers 
nwithn < 101.] 
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47. 


48. 


* 49, 


* 50. 


* 51. 


Is this proof that 
1 1 1 3 1 


Po 2a Cae 2 n’ 


whenever n is a positive integer, correct? Justify your 
answer. 


Basis step: The result is true when n = 1 because 


1 3 1 
D272 T 
Inductive step: Assume that the result is true for n. Then 

car ae oe 

1-2 2-3 (n= l)n n(n+1) 
3 1 1 1 
2 n aa 
_ 3 1 
T2 n+l 


Hence, the result is true for n + 1 if it is true for n. This 
completes the proof. 


Suppose that A,, A>, ...,A,, are a collection of sets. Sup- 
pose that R, =A, ®A, and R, = R} PA, for k= 
3, 4, ...,n. Use mathematical induction to prove that x € 
R,, if and only if x belongs to an odd number of the sets 
A,,A),...,A,. (Recall that S$ @ T is the symmetric dif- 
ference of the sets S and T defined in the preamble to 
Exercise 38 of Section 2.2.) 


Show that n circles divide the plane into n? — n + 2 re- 
gions if every two circles intersect in exactly two points 
and no three circles contain a common point. 


Show that n planes divide three-dimensional space into 
(m? + 5n + 6)/6 regions if any three of these planes have 
exactly one point in common and no four contain a com- 
mon point. 


Use the well-ordering property to show that V2 is ir- 
rational. [Hint: Assume that V2 is rational. Show that 
the set of positive integers of the form by2 has a least 


Links » 
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52. 


53. 


* 54. 


55. 


** 56. 


57. 


58. 


element a. Then show that ay/2 — ais a smaller positive 

integer of this form. ] 

A set is well ordered if every nonempty subset of this 

set has a least element. Determine whether each of the 

following sets is well ordered. 

a) the set of integers 

b) the set of integers greater than —100 

c) the set of positive rationals 

d) the set of positive rationals with denominator less than 
100 

a) Show that if ay, az, .. 
gcd(a, do, ... 
gcd(a,» a). 

b) Use part (a), together with the Euclidean algorithm, 
to develop a recursive algorithm for computing the 
greatest common divisor of a set of n positive inte- 
gers. 


.,d, are positive integers, then 


» yy, aa) = gcd(ay, dy, ..., Ayo, 


Describe a recursive algorithm for writing the greatest 

common divisor of n positive integers as a linear com- 

bination of these integers. 

Find an explicit formula for f(n) if f(1) = 1 and f(n) = 

f(n—1)+2n-— 1 for n>2. Prove your result using 

mathematical induction. 

Give a recursive definition of the set of bit strings that 

contain twice as many Os as Is. 

Let S be the set of bit strings defined recursively by A € S 

and Ox € S, xl € Sif x E€ S, where å is the empty string. 

a) Find all strings in S of length not exceeding five. 

b) Give an explicit description of the elements of S. 

Let S be the set of strings defined recursively by abc € S, 

bac E S, and acb E€ S, where a, b, and c are fixed let- 

ters; and for all x € S, abcx € S; abxc € S, axbc € S, and 

xabc € S, where x is a variable representing a string of 

letters. 

a) Find all elements of S of length eight or less. 

b) Show that every element of S has a length divisible by 
three. 


JOHN McCARTHY (1927-2011) John McCarthy was born in Boston. He grew up in Boston and in Los An- 
geles. He studied mathematics as both an undergraduate and a graduate student, receiving his B.S. in 1948 from 
the California Institute of Technology and his Ph.D. in 1951 from Princeton. After graduating from Princeton, 
McCarthy held positions at Princeton, Stanford, Dartmouth, and M.LT. He held a position at Stanford from 1962 
until 1994, and was an emeritus professor there. At Stanford, he was the director of the Artificial Intelligence 
Laboratory, held a named chair in the School of Engineering, and was a senior fellow at the Hoover Institution. 

McCarthy was a pioneer in the study of artificial intelligence, a term he coined in 1955. He worked 
on problems related to the reasoning and information needs required for intelligent computer behavior. 
McCarthy was among the first computer scientists to design time-sharing computer systems. He devel- 
oped LISP, a programming language for computing using symbolic expressions. He played an important 


role in using logic to verify the correctness of computer programs. McCarthy has also worked on the so- 
cial implications of computer technology. In his later years he worked on the problem of how people and 
computers make conjectures through assumptions that complications are absent from situations. McCarthy was an advocate of the 
sustainability of human progress and was optimistic about the future of humanity. In his later years he also wrote science fiction 
stories. Some of his last work explored the possibility that the world is a computer program written by some higher force. 
Among the awards McCarthy won are the Turing Award from the Association for Computing Machinery, the Research Excel- 
lence Award of the International Conference on Artificial Intelligence, the Kyoto Prize, and the National Medal of Science. 


The set B of all balanced strings of parentheses is defined 

recursively by A € B, where A is the empty string; (x) € B, 

xy € Bifx yes. 

59. Show that (()()) is a balanced string of parentheses and 
(())) is not a balanced string of parentheses. 


60. Find all balanced strings of parentheses with exactly six 
symbols. 


61. Find all balanced strings of parentheses with four or fewer 
symbols. 


62. Use induction to show that if x is a balanced string of 
parentheses, then the number of left parentheses equals 
the number of right parentheses in x. 


Define the function N on the set of strings of parentheses by 


N(A) =0,N(Q) = 1, NỌ) = -1, 
N(uv) = N(u) + N(v), 
where À is the empty string, and u and v are strings. It can be 
shown that N is well defined. 
63. Find 
a) N(()). b) NOOO. 
© N(O@). d) NO(O)O)). 


** 64. Show that a string w of parentheses is balanced if and 
only if N(w) = 0 and N(u) > 0 whenever u is a prefix of 
w, that is, w = uv. 


*65. Give arecursive algorithm for finding all balanced strings 
of parentheses containing n or fewer symbols. 


66. Give a recursive algorithm for finding gcd(a, b), where 
a and b are nonnegative integers not both zero, 
based on these facts: gcd(a, b) = gcd(b,a) if a> b, 
gced(0, b) = b, gcd(a, b) = 2 gcd(a/2, b/2) if a and b are 
even, gcd(a, b) = gcd(a/2, b) if a is even and b is odd, 
and gcd(a, b) = gcd(a, b — a). 

67. Verify the program segment 

if x > y then 
X=} 
with respect to the initial assertion T and the final asser- 
tion x < y. 
*68. Develop a rule of inference for verifying recursive pro- 
grams and use it to verify the recursive algorithm for com- 
puting factorials given as Algorithm 1 in Section 5.4. 


Computer Projects 


Write programs with these input and output. 


** 1, Given a 2” x 2” checkerboard with one square missing, 
construct a tiling of this checkerboard using right triomi- 
noes. 


** 2. Generate all well-formed formulae for expressions in- 
volving the variables x, y, and z and the operators {+, *, 
/, —} with n or fewer symbols. 


**3, Generate all well-formed formulae for propositions with 
n or fewer symbols where each symbol is T, F, one of 
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69. Devise a recursive algorithm that counts the number of 
times the integer 0 occurs in a list of integers. 

Exercises 70-77 deal with some unusual sequences, in- 

formally called self-generating sequences, produced by 

simple recurrence relations or rules. In particular, Exer- 

cises 70-75 deal with the sequence {a(n)} defined by 

a(n) =n—a(a(n—1)) for n> 1 and a(0)= 0. (This se- 

quence, as well as those in Exercises 74 and 75, are defined 

in Douglas Hofstader’s fascinating book Gédel, Escher, Bach 

([H099]). 

70. Find the first 10 terms of the sequence {a(n)} defined in 
the preamble to this exercise. 

*71. Prove that this sequence is well defined. That is, show that 

a(n) is uniquely defined for all nonnegative integers n. 


**72. Prove that a(n) = |(n + l)u] where u = (-1 + /5)/2. 


[Hint: First show for all n > 0 that (un — |wn|) + G2n— 
Lu?n]) = 1. Then show for all real numbers a with 0 < 
a < landa #1- n that |1+wd-a@]|+la+yp| = 
1, considering the cases 0 < «œ < 1 — p and 1 — u < œ < 
1 separately.] 
*73. Use the formula from Exercise 72 to show that 

a(n)=a(n—1) if pun-—|un|<l-pu and a(n)= 
a(n — 1) + 1 otherwise. 

74. Find the first 10 terms of each of the following self- 
generating sequences: 
a) a(n) = n — a(a(a(n — 1))) for n > 1, a(0) = 0 
b) a(n) = n — a(a(a(a(n — 1)))) for n > 1, a(0) = 0 
c) a(n) =a(n—a(l(n-— 1))+ a(n -a(n — 2)) for n> 3, 

a(1) = 1 and a(2) = 1 

75. Find the first 10 terms of both the sequences m(n) and f (n) 
defined by the following pair of interwoven recurrence re- 
lations: m(n) = n — f(m(n — 1)), f(n) =n — m(f(n — 1)) 
for n > 1, f(0) = 1 and m(0) = 0. 

Golomb’s self-generating sequence is the unique nonde- 

creasing sequence of positive integers a,, dj, a3, ... that has 

the property that it contains exactly a, occurrences of k for 

each positive integer k. 

76. Find the first 20 terms of Golomb’s self-generating se- 
quence. 

*77, Show that if f(n) is the largest integer m such that a,, = n, 

where a,, is the mth term of Golomb’s self-generating se- 


quence, then f(n) = X; 1 a, and f(f(n)) = D, kay. 


the propositional variables p and q, or an operator from 
{3,V,A, >, o}. 


4. Given a string, find its reversal. 


5. Given a real number a and a nonnegative integer n, find 
a” using recursion. 


6. Given a real number a and a nonnegative integer n, find 
an ; 
a° using recursion. 


404 5/ Induction and Recursion 


ET, 


10. 


11. 


Given a real number a and a nonnegative integer n, find a” 


using the binary expansion of n and a recursive algorithm 
for computing a”. 


. Given two integers not both zero, find their greatest com- 


mon divisor using recursion. 


. Given a list of integers and an element x, locate x in this 


list using a recursive implementation of a linear search. 


Given a list of integers and an element x, locate x in this 
list using a recursive implementation of a binary search. 


Given a nonnegative integer n, find the nth Fibonacci 
number using iteration. 


Computations and Explorations 


12. 


13. 


14. 


15. 


Given a nonnegative integer n, find the nth Fibonacci 
number using recursion. 


Given a positive integer, find the number of partitions of 
this integer. (See Exercise 49 of Section 5.3.) 


Given positive integers m and n, find A(m, n), the value of 
Ackermann’s function at the pair (m, n). (See the pream- 
ble to Exercise 50 of Section 5.3.) 


Given a list of integers, sort these integers using the merge 
sort. 


Use a computational program or programs you have written to do these exercises. 


. What are the largest values of n for which n! has fewer than 


100 decimal digits and fewer than 1000 decimal digits? 


. Determine which Fibonacci numbers are divisible by 5, 


which are divisible by 7, and which are divisible by 11. 
Prove that your conjectures are correct. 


. Construct tilings using right triominoes of various 16 x 


16, 32 x 32, and 64 x 64 checkerboards with one square 
missing. 


. Explore which mX n checkerboards can be completely 


covered by right triominoes. Can you make a conjecture 
that answers this question? 


Writing Projects 


Respond to these with essays using outside sources. 


. Describe the origins of mathematical induction. Who were 


the first people to use it and to which problems did they 
apply it? 


. Explain how to prove the Jordan curve theorem for sim- 


ple polygons and describe an algorithm for determining 
whether a point is in the interior or exterior of a simple 


polygon. 


. Describe how the triangulation of simple polygons is used 


in some key algorithms in computational geometry. 


. Describe a variety of different applications of the Fi- 


bonacci numbers to the biological and the physical 
sciences. 


**5 


**6 


. Implement an algorithm for determining whether a point 


is in the interior or exterior of a simple polygon. 


. Implement an algorithm for triangulating a simple 


polygon. 


. Which values of Ackermann’s function are small enough 


that you are able to compute them? 


. Compare either the number of operations or the time 


needed to compute Fibonacci numbers recursively versus 
that needed to compute them iteratively. 


. Discuss the uses of Ackermann’s function both in the the- 


ory of recursive definitions and in the analysis of the com- 
plexity of algorithms for set unions. 


. Give the recursive definition of Knuth’s up-arrow notation 


and illustrate its use with a variety of examples, includ- 
ing how it can be used to express values of the Acker- 
mann function (defined in the preamble of Exercise 50 in 
Section 5.3). 


. Discuss some of the various methodologies used to es- 


tablish the correctness of programs and compare them to 
Hoare’s methods described in Section 5.5. 


. Explain how the ideas and concepts of program correct- 


ness can be extended to prove that operating systems are 
secure. 


6.1 The Basics of 
Counting 


6.2 The Pigeonhole 
Principle 


6.3 Permutations 
and 
Combinations 


6.4 Binomial 
Coefficients 
and Identities 


6.5 Generalized 
Permutations 
and 
Combinations 


6.6 Generating 
Permutations 
and 
Combinations 


Counting 


C ombinatorics, the study of arrangements of objects, is an important part of discrete math- 
ematics. This subject was studied as long ago as the seventeenth century, when combina- 
torial questions arose in the study of gambling games. Enumeration, the counting of objects with 
certain properties, is an important part of combinatorics. We must count objects to solve many 
different types of problems. For instance, counting is used to determine the complexity of algo- 
rithms. Counting is also required to determine whether there are enough telephone numbers or 
Internet protocol addresses to meet demand. Recently, it has played a key role in mathematical 
biology, especially in sequencing DNA. Furthermore, counting techniques are used extensively 
when probabilities of events are computed. 

The basic rules of counting, which we will study in Section 6.1, can solve a tremendous 
variety of problems. For instance, we can use these rules to enumerate the different telephone 
numbers possible in the United States, the allowable passwords on a computer system, and the 
different orders in which the runners in a race can finish. Another important combinatorial tool 
is the pigeonhole principle, which we will study in Section 6.2. This states that when objects are 
placed in boxes and there are more objects than boxes, then there is a box containing at least two 
objects. For instance, we can use this principle to show that among a set of 15 or more students, 
at least 3 were born on the same day of the week. 

We can phrase many counting problems in terms of ordered or unordered arrangements of 
the objects of a set with or without repetitions. These arrangements, called permutations and 
combinations, are used in many counting problems. For instance, suppose the 100 top finishers 
on a competitive exam taken by 2000 students are invited to a banquet. We can count the possible 
sets of 100 students that will be invited, as well as the ways in which the top 10 prizes can be 
awarded. 

Another problem in combinatorics involves generating all the arrangements of a specified 
kind. This is often important in computer simulations. We will devise algorithms to generate 
arrangements of various types. 


The Basics of Counting 


6.1.1 Introduction 


Suppose that a password on a computer system consists of six, seven, or eight characters. Each 
of these characters must be a digit or a letter of the alphabet. Each password must contain at least 
one digit. How many such passwords are there? The techniques needed to answer this question 
and a wide variety of other counting problems will be introduced in this section. 

Counting problems arise throughout mathematics and computer science. For example, we 
must count the successful outcomes of experiments and all the possible outcomes of these exper- 
iments to determine probabilities of discrete events. We need to count the number of operations 
used by an algorithm to study its time complexity. 

We will introduce the basic techniques of counting in this section. These methods serve as 
the foundation for almost all counting techniques. 
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EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


6.1.2 Basic Counting Principles 


We first present two basic counting principles, the product rule and the sum rule. Then we 
will show how they can be used to solve many different counting problems. 
The product rule applies when a procedure is made up of separate tasks. 


THE PRODUCT RULE Suppose that a procedure can be broken down into a sequence of 
two tasks. If there are n; ways to do the first task and for each of these ways of doing the first 
task, there are n, ways to do the second task, then there are n,n, ways to do the procedure. 


Examples 1—10 show how the product rule is used. 


A new company with just two employees, Sanchez and Patel, rents a floor of a building with 
12 offices. How many ways are there to assign different offices to these two employees? 


Solution: The procedure of assigning offices to these two employees consists of assigning an 
office to Sanchez, which can be done in 12 ways, then assigning an office to Patel different from 
the office assigned to Sanchez, which can be done in 11 ways. By the product rule, there are 
12. 11 = 132 ways to assign offices to these two employees. < 


The chairs of an auditorium are to be labeled with an uppercase English letter followed by a 
positive integer not exceeding 100. What is the largest number of chairs that can be labeled 
differently? 


Solution: The procedure of labeling a chair consists of two tasks, namely, assigning to the seat 
one of the 26 uppercase English letters, and then assigning to it one of the 100 possible integers. 
The product rule shows that there are 26 - 100 = 2600 different ways that a chair can be labeled. 
Therefore, the largest number of chairs that can be labeled differently is 2600. < 


There are 32 computers in a data center in the cloud. Each of these computers has 24 ports. How 
many different computer ports are there in this data center? 


Solution: The procedure of choosing a port consists of two tasks, first picking a computer and 
then picking a port on this computer. Because there are 32 ways to choose the computer and 24 
ways to choose the port no matter which computer has been selected, the product rule shows 
that there are 32 - 24 = 768 ports. < 


An extended version of the product rule is often useful. Suppose that a procedure is carried 
out by performing the tasks 7), T}, ..., T in sequence. If each task T;, i = 1, 2, ...,, can be 
done in n; ways, regardless of how the previous tasks were done, then there aren, + n+ + -Am 
ways to carry out the procedure. This version of the product rule can be proved by mathematical 
induction from the product rule for two tasks (see Exercise 76). 


How many different bit strings of length seven are there? 


Solution: Each of the seven bits can be chosen in two ways, because each bit is either 0 or 1. 
Therefore, the product rule shows there are a total of 2’ = 128 different bit strings of length 
seven. < 


EXAMPLE 5 


26 choices 10 choices 
for each for each 
letter digit 


EXAMPLE 6 


EXAMPLE 7 


Counting the number of 
onto functions is harder. 
We'll do this in 

Chapter 8. 


EXAMPLE 8 


Links » 


Current projections are 
that by 2038, it will be 
necessary to add one or 
more digits to North 
American telephone 
numbers. 
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How many different license plates can be made if each plate contains a sequence of three up- 
percase English letters followed by three digits (and no sequences of letters are prohibited, even 
if they are obscene)? 


Solution: There are 26 choices for each of the three uppercase English letters and 10 choices for 


each of the three digits. Hence, by the product rule there are a total of 26 - 26 - 26 - 10 - 10 - 10 = 
17,576,000 possible license plates. < 


Counting Functions How many functions are there from a set with m elements to a set with 
n elements? 


Solution: A function corresponds to a choice of one of the n elements in the codomain for each of 


the m elements in the domain. Hence, by the product rule there aren-n- = -n = n” functions 
from a set with m elements to one with n elements. For example, there are 5° = 125 different 
functions from a set with three elements to a set with five elements. < 


Counting One-to-One Functions How many one-to-one functions are there from a set with 
m elements to one with n elements? 


Solution: First note that when m > n there are no one-to-one functions from a set with m ele- 
ments to a set with n elements. 

Now let m < n. Suppose the elements in the domain are aj, az, ..., Am. There are n ways 
to choose the value of the function at a}. Because the function is one-to-one, the value of the 
function at a, can be picked in n — 1 ways (because the value used for a, cannot be used again). 
In general, the value of the function at a, can be chosen in n — k + 1 ways. By the product rule, 
there are n(n — 1)(n — 2) --- (n — m + 1) one-to-one functions from a set with m elements to one 
with n elements. 

For example, there are 5 - 4 - 3 = 60 one-to-one functions from a set with three elements to 
a set with five elements. < 


The Telephone Numbering Plan The North American numbering plan (NANP) specifies the 
format of telephone numbers in the U.S., Canada, and many other parts of North America. A 
telephone number in this plan consists of 10 digits, which are split into a three-digit area code, a 
three-digit office code, and a four-digit station code. Because of signaling considerations, there 
are certain restrictions on some of these digits. To specify the allowable format, let X denote 
a digit that can take any of the values 0 through 9, let N denote a digit that can take any of 
the values 2 through 9, and let Y denote a digit that must be a 0 or a 1. Two numbering plans, 
which will be called the old plan, and the new plan, will be discussed. (The old plan, in use in 
the 1960s, has been replaced by the new plan, but the recent rapid growth in demand for new 
numbers for mobile phones and devices will eventually make even this new plan obsolete. In 
this example, the letters used to represent digits follow the conventions of the North American 
Numbering Plan.) As will be shown, the new plan allows the use of more numbers. 

In the old plan, the formats of the area code, office code, and station code are NYX, NNX, and 
XXXX, respectively, so that telephone numbers had the form NYX-NNX-XXXX. In the new plan, 
the formats of these codes are NXX, NXX, and XXXX, respectively, so that telephone numbers 
have the form NXX-NXX-XXXX. How many different North American telephone numbers are 
possible under the old plan and under the new plan? 


Solution: By the product rule, there are 8-2-10= 160 area codes with format NYX and 
8-10-10 = 800 area codes with format NXX. Similarly, by the product rule, there are 
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Note that we have 
ignored restrictions that 
rule out N11 station 
codes for most area 
codes. 


EXAMPLE 9 


EXAMPLE 10 


8- 8-10 = 640 office codes with format NNX. The product rule also shows that there are 
10- 10-10-10 = 10,000 station codes with format XXXX. 
Consequently, applying the product rule again, it follows that under the old plan there are 
160 - 640 - 10,000 = 1,024,000,000 
different numbers available in North America. Under the new plan, there are 


800 - 800 - 10,000 = 6,400,000,000 


different numbers available. < 


What is the value of k after the following code, where n,, m, ..., n, are positive integers, has 
been executed? 


k:=0 
for i, := 1 ton, 
for i, := 1 ton, 


for i,, := 1 ton, 


m 


k:=k+1 


Solution: The initial value of k is zero. Each time the nested loop is traversed, 1 is added 
to k. Let T; be the task of traversing the ith loop. Then the number of times the loop is tra- 
versed is the number of ways to do the tasks T}, T, ..., T,- The number of ways to carry out 
the task T; j= 1,2, ...,m, is n;, because the jth loop is traversed once for each integer ij with 
1 <i, <n,. By the product rule, it follows that the nested loop is traversed n,n, ---n,, times. 
Hence, the final value of k is nym + np. < 


Counting Subsets of a Finite Set Use the product rule to show that the number of different 
subsets of a finite set S is 215l, 


Solution: Let S be a finite set. List the elements of S in arbitrary order. Recall from 
Section 2.2 that there is a one-to-one correspondence between subsets of S and bit strings of 
length ||. Namely, a subset of S is associated with the bit string with a 1 in the ith position if 
the ith element in the list is in the subset, and a 0 in this position otherwise. By the product rule, 
there are 2/5! bit strings of length |S]. Hence, |P(S)| = 2!5!. (Recall that we used mathematical 
induction to prove this fact in Example 10 of Section 5.1.) < 


The product rule is often phrased in terms of sets in this way: If A,, A>, ...,A,, are finite 
sets, then the number of elements in the Cartesian product of these sets is the product of the 
number of elements in each set. To relate this to the product rule, note that the task of choos- 
ing an element in the Cartesian product A, X A, X = XA,, is done by choosing an element 
in A,, an element in A), ..., and an element in A,,. By the product rule it follows that 

lA, x A, os 208 x Anl = |A,| $ |A,| a ae eae |A 


al 


EXAMPLE 11 


Soon it won't be that 
costly to have your own 
genetic code found. 


EXAMPLE 12 
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DNA and Genomes The hereditary information of a living organism is encoded using de- 
oxyribonucleic acid (DNA), or in certain viruses, ribonucleic acid (RNA). DNA and RNA are 
extremely complex molecules, with different molecules interacting in a vast variety of ways to 
enable living process. For our purposes, we give only the briefest description of how DNA and 
RNA encode genetic information. 

DNA molecules consist of two strands consisting of blocks known as nucleotides. Each 
nucleotide contains subcomponents called bases, each of which is adenine (A), cytosine (C), 
guanine (G), or thymine (T). The two strands of DNA are held together by hydrogen bonds con- 
necting different bases, with A bonding only with T, and C bonding only with G. Unlike DNA, 
RNA is single stranded, with uracil (U) replacing thymine as a base. So, in DNA the possible 
base pairs are A-T and C-G, while in RNA they are A-U, and C-G. The DNA of a living crea- 
ture consists of multiple pieces of DNA forming separate chromosomes. A gene is a segment 
of a DNA molecule that encodes a particular protein. The entirety of genetic information of an 
organism is called its genome. 

Sequences of bases in DNA and RNA encode long chains of proteins called amino acids. 
There are 22 essential amino acids for human beings. We can quickly see that a sequence of 
at least three bases are needed to encode these 22 different amino acid. First note, that because 
there are four possibilities for each base in DNA, A, C, G, and T, by the product rule there are 
4? = 16 < 22 different sequences of two bases. However, there are 4° = 64 different sequences 
of three bases, which provide enough different sequences to encode the 22 different amino acids 
(even after taking into account that several different sequences of three bases encode the same 
amino acid). 

The DNA of simple living creatures such as algae and bacteria have between 10° and 107 
links, where each link is one of the four possible bases. More complex organisms, such as in- 
sects, birds, and mammals, have between 108 and 10!° links in their DNA. So, by the product 
rule, there are at least 4!° different sequences of bases in the DNA of simple organisms and at 
least 4! different sequences of bases in the DNA of more complex organisms. These are both 
incredibly huge numbers, which helps explain why there is such tremendous variability among 
living organisms. In the past several decades techniques have been developed for determining 
the genome of different organisms. The first step is to locate each gene in the DNA of an organ- 
ism. The next task, called gene sequencing, is the determination of the sequence of links on 
each gene. (The specific sequence of links on these genes depends on the particular individual 
representative of a species whose DNA is analyzed.) For example, the human genome includes 
approximately 23,000 genes, each with 1000 or more links. Gene sequencing techniques take 
advantage of many recently developed algorithms and are based on numerous new ideas in com- 
binatorics. Many mathematicians and computer scientists work on problems involving genomes, 
taking part in the fast moving fields of bioinformatics and computational biology. < 


We now introduce the sum rule. 


THE SUM RULE Ifa task can be done either in one of n; ways or in one of n, ways, where 
none of the set of n) ways is the same as any of the set of n, ways, then there are n; + n 
ways to do the task. 


Example 12 illustrates how the sum rule is used. 


Suppose that either a member of the mathematics faculty or a student who is a mathematics ma- 
jor is chosen as a representative to a university committee. How many different choices are there 
for this representative if there are 37 members of the mathematics faculty and 83 mathematics 
majors and no one is both a faculty member and a student? 


Solution: There are 37 ways to choose a member of the mathematics faculty and there are 83 
ways to choose a student who is a mathematics major. Choosing a member of the mathematics 
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EXAMPLE 13 


EXAMPLE 14 


faculty is never the same as choosing a student who is a mathematics major because no one is 
both a faculty member and a student. By the sum rule it follows that there are 37 + 83 = 120 
possible ways to pick this representative. < 


We can extend the sum rule to more than two tasks. Suppose that a task can be done in one 
of n; ways, in one of n, ways, ... , or in one of n,, ways, where none of the set of n; ways of 
doing the task is the same as any of the set of n, ways, for all pairs i and j with 1 < i < j < m. 
Then the number of ways to do the task is n; +n, + © +n,,. This extended version of the 
sum rule is often useful in counting problems, as Examples 13 and 14 show. This version of 
the sum rule can be proved using mathematical induction from the sum rule for two sets. (See 
Exercise 75.) 


A student can choose a computer project from one of three lists. The three lists contain 23, 15, 
and 19 possible projects, respectively. No project is on more than one list. How many possible 
projects are there to choose from? 


Solution: The student can choose a project by selecting a project from the first list, the second 
list, or the third list. Because no project is on more than one list, by the sum rule there are 
23 + 15 + 19 = 57 ways to choose a project. < 


What is the value of k after the following code, where n,, m, ...,,, are positive integers, has 
been executed? 


k:=0 

for i, := 1 ton, 
k:=k+1 

for i, := 1 to m 
k:=k+1 


for i„ := 1 to n, 
k:=k+1 


Solution: The initial value of k is zero. This block of code is made up of m different loops. 
Each time a loop is traversed, 1 is added to k. To determine the value of k after this code has 
been executed, we need to determine how many times we traverse a loop. Note that there are 
n; ways to traverse the ith loop. Because we only traverse one loop at a time, the sum rule 
shows that the final value of k, which is the number of ways to traverse one of the m loops is 
ni +m +e +m < 


The sum rule can be phrased in terms of sets as: If Aj, A>, ...,A,, are pairwise disjoint 
finite sets, then the number of elements in the union of these sets is the sum of the numbers of 
elements in the sets. To relate this to our statement of the sum rule, note there are |A;| ways to 
choose an element from A; for i = 1, 2, ..., m. Because the sets are pairwise disjoint, when we 
select an element from one of the sets A;, we do not also select an element from a different set 
A;. Consequently, by the sum rule, because we cannot select an element from two of these sets 
at the same time, the number of ways to choose an element from one of the sets, which is the 
number of elements in the union, is 


|A; UA, U e VA, | = |A| + [Ag] + + + JAn | when A; NA; = for alli, j. 
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This equality applies only when the sets in question are pairwise disjoint. The situation is much 
more complicated when these sets have elements in common. That situation will be briefly 
discussed later in this section and discussed in more depth in Chapter 8. 


6.1.3 More Complex Counting Problems 


Many counting problems cannot be solved using just the sum rule or just the product rule. 
However, many complicated counting problems can be solved using both of these rules in com- 
bination. We begin by counting the number of variable names in the programming language 
BASIC. (In the exercises, we consider the number of variable names in JAVA.) Then we will 
count the number of valid passwords subject to a particular set of restrictions. 


In a version of the computer language BASIC, the name of a variable is a string of one or 
two alphanumeric characters, where uppercase and lowercase letters are not distinguished. (An 
alphanumeric character is either one of the 26 English letters or one of the 10 digits.) Moreover, 
a variable name must begin with a letter and must be different from the five strings of two 
characters that are reserved for programming use. How many different variable names are there 
in this version of BASIC? 


Solution: Let V equal the number of different variable names in this version of BASIC. Let V, 
be the number of these that are one character long and V, be the number of these that are two 
characters long. Then by the sum rule, V = V, + V3. Note that V, = 26, because a one-character 
variable name must be a letter. Furthermore, by the product rule there are 26 - 36 strings of 
length two that begin with a letter and end with an alphanumeric character. However, five of 
these are excluded, so V, = 26 - 36 — 5 = 931. Hence, there are V = V} + V, = 26 + 931 = 957 
different names for variables in this version of BASIC. < 


Each user on a computer system has a password, which is six to eight characters long, where 
each character is an uppercase letter or a digit. Each password must contain at least one digit. 
How many possible passwords are there? 


Solution: Let P be the total number of possible passwords, and let Ps, P}, and Pg denote the 
number of possible passwords of length 6, 7, and 8, respectively. By the sum rule, P = Pe + 
P} + Ps. We will now find Pg, P}, and P}. Finding Pe directly is difficult. To find P, it is easier 
to find the number of strings of uppercase letters and digits that are six characters long, including 
those with no digits, and subtract from this the number of strings with no digits. By the product 
rule, the number of strings of six characters is 36°, and the number of strings with no digits is 
26°. Hence, 


P; = 36° — 26° = 2,176,782,336 — 308,915,776 = 1,867,866,560. 
Similarly, we have 
P, = 36’ — 26’ = 78,364, 164,096 — 8,031,810,176 = 70,332,353,920 


and 


P; = 36° — 268 = 2,821, 109,907,456 — 208,827,064,576 
= 2,612,282, 842,880. 


Consequently, 


P = P +P, + Ps = 2,684,483,063,360. <4 
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The lack of available 
IPv4 address has 
become a crisis! 


Bit Number 0 1 2 3 4 8 16 24 31 
Class A 0 netid hostid 

Class B 1] 0 netid hostid 

Class C 1 1 0 netid hostid 

Class D 1 1 1 0 Multicast Address 

Class E 1 1 1 1 0 Address 


FIGURE 1 Internet addresses (IPv4). 


Counting Internet Addresses In the Internet, which is made up of interconnected physical 
networks of computers, each computer (or more precisely, each network connection of a com- 
puter) is assigned an Internet address. In Version 4 of the Internet Protocol (IPv4), still in use 
today, an address is a string of 32 bits. It begins with a network number (netid ). The netid is 
followed by a host number (hostid ), which identifies a computer as a member of a particular 
network. 

Three forms of addresses are used, with different numbers of bits used for netids and hostids. 
Class A addresses, used for the largest networks, consist of 0, followed by a 7-bit netid and a 
24-bit hostid. Class B addresses, used for medium-sized networks, consist of 10, followed by 
a 14-bit netid and a 16-bit hostid. Class C addresses, used for the smallest networks, consist 
of 110, followed by a 21-bit netid and an 8-bit hostid. There are several restrictions on ad- 
dresses because of special uses: 1111111 is not available as the netid of a Class A network, 
and the hostids consisting of all Os and all 1s are not available for use in any network. A com- 
puter on the Internet has either a Class A, a Class B, or a Class C address. (Besides Class A, 
B, and C addresses, there are also Class D addresses, reserved for use in multicasting when 
multiple computers are addressed at a single time, consisting of 1110 followed by 28 bits, and 
Class E addresses, reserved for future use, consisting of 11110 followed by 27 bits. Neither 
Class D nor Class E addresses are assigned as the IPv4 address of a computer on the Internet.) 
Figure 1 illustrates IPv4 addressing. (Limitations on the number of Class A and Class B netids 
have made IPv4 addressing inadequate; IPv6, a new version of IP, uses 128-bit addresses to 
solve this problem.) 

How many different IPv4 addresses are available for computers on the Internet? 


Solution: Let x be the number of available addresses for computers on the Internet, and let x4, 
Xp, and xç denote the number of Class A, Class B, and Class C addresses available, respectively. 
By the sum rule, x = x4 + Xg + xc- 

To find x,, note that there are 27 — 1 = 127 Class A netids, recalling that the netid 
1111111 is unavailable. For each netid, there are 22* — 2 = 16,777,214 hostids, recalling that the 
hostids consisting of all Os and all 1s are unavailable. Consequently, x, = 127 - 16,777,214 = 
2,130,706, 178. 

To find xg and xç, note that there are 2'4 = 16,384 Class B netids and 27! = 2,097,152 
Class C netids. For each Class B netid, there are 2!6 — 2 = 65,534 hostids, and for each 
Class C netid, there are 28 — 2 = 254 hostids, recalling that in each network the hostids 
consisting of all Os and all 1s are unavailable. Consequently, x, = 1,073,709,056 and xç = 
532,676,608. 

We conclude that the total number of IPv4 addresses available is x =x, +x, + Xç = 
2,130,706, 178 + 1,073,709,056 + 532,676,608 = 3,737,091,842. 


6.1.4 The Subtraction Rule (Inclusion—Exclusion for Two Sets) 


Suppose that a task can be done in one of two ways, but some of the ways to do it are common 
to both ways. In this situation, we cannot use the sum rule to count the number of ways to do 


Overcounting is perhaps 
the most common 
enumeration error. 
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the task. If we add the number of ways to do the tasks in these two ways, we get an overcount 
of the total number of ways to do it, because the ways to do the task that are common to the two 
ways are counted twice. 


To correctly count the number of ways to do the two tasks, we must subtract the number of 
ways that are counted twice. This leads us to an important counting rule. 


THE SUBTRACTION RULE If a task can be done in either n; ways or n, ways, then the 
number of ways to do the task is n; + n, minus the number of ways to do the task that are 
common to the two different ways. 


The subtraction rule is also known as the principle of inclusion—exclusion, especially when 
it is used to count the number of elements in the union of two sets. Suppose that A, and A, are 
sets. Then, there are |A,| ways to select an element from A, and |A,| ways to select an element 
from A,. The number of ways to select an element from A, or from A), that is, the number of 
ways to select an element from their union, is the sum of the number of ways to select an element 
from A, and the number of ways to select an element from A,, minus the number of ways to 
select an element that is in both A, and Ay. Because there are |A; U A, | ways to select an element 
in either A, or in A,, and |A, N A,| ways to select an element common to both sets, we have 


|A; UA,| = JA, | JP |A,| = |A; NA,|. 


This is the formula given in Section 2.2 for the number of elements in the union of two sets. 
Example 18 illustrates how we can solve counting problems using the subtraction principle. 


How many bit strings of length eight either start with a 1 bit or end with the two bits 00? 


Solution: Figure 2 illustrates the three counting problems we need to solve before we 
can apply the principle of inclusion—exclusion. We can construct a bit string of length 
eight that either starts with a 1 bit or ends with the two bits 00, by constructing a bit 
string of length eight beginning with a 1 bit or by constructing a bit string of length 
eight that ends with the two bits 00. We can construct a bit string of length eight that 
begins with a 1 in 2’=128 ways. This follows by the product rule, because the first 
bit can be chosen in only one way and each of the other seven bits can be chosen in 
two ways. Similarly, we can construct a bit string of length eight ending with the two 
bits 00, in 2° = 64 ways. This follows by the product rule, because each of the first six bits 
can be chosen in two ways and the last two bits can be chosen in only one way. 

Some of the ways to construct a bit string of length eight starting with a 1 are the same 
as the ways to construct a bit string of length eight that ends with the two bits 00. There are 
2° = 32 ways to construct such a string. This follows by the product rule, because the first 
bit can be chosen in only one way, each of the second through the sixth bits can be chosen 
in two ways, and the last two bits can be chosen in one way. Consequently, the number of 
bit strings of length eight that begin with a 1 or end with a 00, which equals the number 
of ways to construct a bit string of length eight that begins with a 1 or that ends with 00, 
equals 128 + 64 — 32 = 160. < 


We present an example that illustrates how the formulation of the principle of inclusion— 
exclusion can be used to solve counting problems. 


A computer company receives 350 applications from college graduates for a job planning a line 
of new web servers. Suppose that 220 of these applicants majored in computer science, 147 
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|A; UAg| = |U] — |A; U Ahl 
= |U] - (|A;] + |A2| — |A; N A21) 
= 350 — (220 + 147-51) 
= 350 - 316 
=34 


|A,|=220 |4 nA |=51_ |A| = 147 


FIGURE 3 Applicants who majored in neither computer science nor business. 


majored in business, and 51 majored both in computer science and in business. How many of 
these applicants majored neither in computer science nor in business? 


Solution: To find the number of these applicants who majored neither in computer science nor 
in business, we can subtract the number of students who majored either in computer science 
or in business (or both) from the total number of applicants. Let A, be the set of students who 
majored in computer science and A, the set of students who majored in business. Then A, U A, 
is the set of students who majored in computer science or business (or both), and A, N A, is the 
set of students who majored both in computer science and in business. By the subtraction rule 
the number of students who majored either in computer science or in business (or both) equals 


|A, UA] = |A;| + |42| — |A; N.Ay| = 220 + 147 — 51 = 316. 


We conclude that 350 — 316 = 34 of the applicants majored neither in computer science nor in 
business. A Venn diagram for this example is shown in Figure 3. < 


The subtraction rule, or the principle of inclusion—exclusion, can be generalized to find the 
number of ways to do one of n different tasks or, equivalently, to find the number of elements 
in the union of n sets, whenever n is a positive integer. We will study the inclusion—exclusion 
principle and some of its many applications in Chapter 8. 


6.1.5 The Division Rule 


We have introduced the product, sum, and subtraction rules for counting. You may wonder 
whether there is also a division rule for counting. In fact, there is such a rule, which can be 
useful when solving certain types of enumeration problems. 


THE DIVISION RULE There are n/d ways to do a task if it can be done using a procedure 
that can be carried out in n ways, and for every way w, exactly d of the n ways correspond to 
way w. 


We can restate the division rule in terms of sets: “If the finite set A is the union of n pairwise 
disjoint subsets each with d elements, then n = |A|/d.” 

We can also formulate the division rule in terms of functions: “If f is a function from A 
to B where A and B are finite sets, and that for every value y € B there are exactly d values 
x E A such that f(x) = y (in which case, we say that f is d-to-one), then |B| = |A|/d.” 


Remark: The division rule comes in handy when it appears that a task can be done in n different 
ways, but it turns out that for each way of doing the task, there are d equivalent ways of doing 
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it. Under these circumstances, we can conclude that there are n/d inequivalent ways of doing 
the task. 


We illustrate the use of the division rule for counting with two examples. 


Suppose that an automated system has been developed that counts the legs of cows in a pasture. 
Suppose that this system has determined that in a farmer’s pasture there are exactly 572 legs. 
How many cows are there is this pasture, assuming that each cow has four legs and that there 
are no other animals present? 


Solution: Let n be the number of cow legs counted in a pasture. Because each cow has four legs, 
by the division rule we know that the pasture contains n/4 cows. Consequently, the pasture with 
572 cow legs has 572/4 = 143 cows in it. < 


How many different ways are there to seat four people around a circular table, where two seat- 
ings are considered the same when each person has the same left neighbor and the same right 
neighbor? 


Solution: We arbitrarily select a seat at the table and label it seat 1. We number the rest of the 
seats in numerical order, proceeding clockwise around the table. Note that are four ways to select 
the person for seat 1, three ways to select the person for seat 2, two ways to select the person 
for seat 3, and one way to select the person for seat 4. Thus, there are 4! = 24 ways to order the 
given four people for these seats. However, each of the four choices for seat 1 leads to the same 
arrangement, as we distinguish two arrangements only when one of the people has a different 
immediate left or immediate right neighbor. Because there are four ways to choose the person 
for seat 1, by the division rule there are 24/4 = 6 different seating arrangements of four people 
around the circular table. 


6.1.6 Tree Diagrams 


Counting problems can be solved using tree diagrams. A tree consists of a root, a number 
of branches leaving the root, and possible additional branches leaving the endpoints of other 
branches. (We will study trees in detail in Chapter 11.) To use trees in counting, we use a branch 
to represent each possible choice. We represent the possible outcomes by the leaves, which are 
the endpoints of branches not having other branches starting at them. 

Note that when a tree diagram is used to solve a counting problem, the number of choices 
of which branch to follow to reach a leaf can vary as in Example 22. 


How many bit strings of length four do not have two consecutive 1s? 


Solution: The tree diagram in Figure 4 displays all bit strings of length four without two con- 
secutive 1s. We see that there are eight bit strings of length four without two consecutive Is. <4 


A playoff between two teams consists of at most five games. The first team that wins three games 
wins the playoff. In how many different ways can the playoff occur? 


Solution: The tree diagram in Figure 5 displays all the ways the playoff can proceed, with the 
winner of each game shown. We see that there are 20 different ways for the playoff to occur. <4 


416 6/Counting 


Winning team 
shown in color 4 4 
oO oO 
© S 
5 B 
© = 
Game 1 
T T T T 
2 2 5 2 
3 5 3 
G a N = Game 2 
T T F T T F 
S S S S S = 
3 3 B 3 3 3 
= © = N = N Game 3 
ha os) 
g oO 
5 F F F T @ F 5 
© S S S 3 S S = 
B B 8 B 8 
an an to a b N Game 4 
=) = = = = =) 
oO oO Q Q oO oO 
= = S S S S 
B z B B B B 
N N = N = Game 5 
5 3 g 35 5 3 5 H g 3 g 
oO oO oO oO oO oO oO oO oO oO oO oO 
S S S S pf S S D S S S S 
B 8 B a B 8 8 = = B B B 
© i= © -E = N — N = N — 


FIGURE 5 Best three games out of five playoffs. 


EXAMPLE 24 Suppose that “I Love New Jersey” T-shirts come in five different sizes: S, M, L, XL, and XXL. 
Further suppose that each size comes in four colors, white, red, green, and black, except for XL, 
which comes only in red, green, and black, and XXL, which comes only in green and black. How 
many different shirts does a souvenir shop have to stock to have at least one of each available 


size and color of the T-shirt? 


Solution: The tree diagram in Figure 6 displays all possible size and color pairs. It follows that 
the souvenir shop owner needs to stock 17 different T-shirts. < 


W = white, R = red, G = green, B = black 


L XL XXL 
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FIGURE 6 Counting varieties of T-shirts. 


Exercises 
1. There are 18 mathematics majors and 325 computer sci- 2. An office building contains 27 floors and has 37 offices 
ence majors at a college. on each floor. How many offices are in the building? 
a) In how many ways can two representatives be picked 3. A multiple-choice test contains 10 questions. There are 
so that one is a mathematics major and the other is a four possible answers for each question. 
computer science major? a) In how many ways can a student answer the questions 
b) In how many ways can one representative be picked on the test if the student answers every question? 
who is either a mathematics major or a computer sci- b) In how many ways can a student answer the questions 
on the test if the student can leave answers blank? 


ence major? 


18. 


19. 


20. 


21. 


. A particular brand of shirt comes in 12 colors, has a male 


version and a female version, and comes in three sizes 
for each sex. How many different types of this shirt are 
made? 


. Six different airlines fly from New York to Denver and 


seven fly from Denver to San Francisco. How many dif- 
ferent pairs of airlines can you choose on which to book 
a trip from New York to San Francisco via Denver, when 
you pick an airline for the flight to Denver and an airline 
for the continuation flight to San Francisco? 


. There are four major auto routes from Boston to Detroit 


and six from Detroit to Los Angeles. How many major 
auto routes are there from Boston to Los Angeles via 
Detroit? 


. How many different three-letter initials can people have? 
. How many different three-letter initials with none of the 


letters repeated can people have? 


. How many different three-letter initials are there that be- 


gin with an A? 


. How many bit strings are there of length eight? 
. How many bit strings of length ten both begin and end 


with a 1? 


. How many bit strings are there of length six or less, not 


counting the empty string? 


. How many bit strings with length not exceeding n, where 


nis a positive integer, consist entirely of 1s, not counting 
the empty string? 


. How many bit strings of length n, where n is a positive 


integer, start and end with 1s? 


. How many strings are there of lowercase letters of length 


four or less, not counting the empty string? 


. How many strings are there of four lowercase letters that 


have the letter x in them? 


. How many strings of five ASCII characters contain the 


character @ (“at” sign) at least once? [Note: There are 

128 different ASCII characters. ] 

How many 5-element DNA sequences 

a) end with A? 

b) start with T and end with G? 

c) contain only A and T? 

d) do not contain C? 

How many 6-element RNA sequences 

a) do not contain U? 

b) end with GU? 

c) start with C? 

d) contain only A or U? 

How many positive integers between 5 and 31 

a) are divisible by 3? Which integers are these? 

b) are divisible by 4? Which integers are these? 

c) are divisible by 3 and by 4? Which integers are these? 

How many positive integers between 50 and 100 

a) are divisible by 7? Which integers are these? 

b) are divisible by 11? Which integers are these? 

c) are divisible by both 7 and 11? Which integers are 
these? 


22. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 
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How many positive integers less than 1000 

a) are divisible by 7? 

b) are divisible by 7 but not by 11? 

c) are divisible by both 7 and 11? 

d) are divisible by either 7 or 11? 

e) are divisible by exactly one of 7 and 11? 

f) are divisible by neither 7 nor 11? 

g) have distinct digits? 

h) have distinct digits and are even? 

How many positive integers between 100 and 999 inclu- 
sive 

a) are divisible by 7? 

b) are odd? 

c) have the same three decimal digits? 

d) are not divisible by 4? 

e) are divisible by 3 or 4? 

f) are not divisible by either 3 or 4? 

g) are divisible by 3 but not by 4? 

h) are divisible by 3 and 4? 

How many positive integers between 1000 and 9999 in- 
clusive 

a) are divisible by 9? 

b) are even? 

c) have distinct digits? 

d) are not divisible by 3? 

e) are divisible by 5 or 7? 

f) are not divisible by either 5 or 7? 

g) are divisible by 5 but not by 7? 

h) are divisible by 5 and 7? 

How many strings of three decimal digits 

a) do not contain the same digit three times? 

b) begin with an odd digit? 

c) have exactly two digits that are 4s? 

How many strings of four decimal digits 

a) do not contain the same digit twice? 

b) end with an even digit? 

c) have exactly three digits that are 9s? 

A committee is formed consisting of one representative 
from each of the 50 states in the United States, where the 
representative from a state is either the governor or one of 
the two senators from that state. How many ways are there 
to form this committee? 

How many license plates can be made using either three 
digits followed by three uppercase English letters or three 
uppercase English letters followed by three digits? 

How many license plates can be made using either two 
uppercase English letters followed by four digits or two 
digits followed by four uppercase English letters? 

How many license plates can be made using either three 
uppercase English letters followed by three digits or four 
uppercase English letters followed by two digits? 

How many license plates can be made using either two 
or three uppercase English letters followed by either two 
or three digits? 
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32. 


33. 


34. 


35. 


36. 


37. 


38. 


39. 


40. 


41. 


42. 


How many strings of eight uppercase English letters are 
there 


a) if letters can be repeated? 

b) if no letter can be repeated? 

c) that start with X, if letters can be repeated? 

d) that start with X, if no letter can be repeated? 

e) that start and end with X, if letters can be repeated? 

f) that start with the letters BO (in that order), if letters 
can be repeated? 

g) that start and end with the letters BO (in that order), 
if letters can be repeated? 

h) that start or end with the letters BO (in that order), if 
letters can be repeated? 


How many strings of eight English letters are there 


a) that contain no vowels, if letters can be repeated? 

b) that contain no vowels, if letters cannot be repeated? 

c) that start with a vowel, if letters can be repeated? 

d) that start with a vowel, if letters cannot be repeated? 

e) that contain at least one vowel, if letters can be re- 
peated? 

f) that contain exactly one vowel, if letters can be re- 
peated? 

g) that start with X and contain at least one vowel, if let- 
ters can be repeated? 

h) that start and end with X and contain at least one 
vowel, if letters can be repeated? 

How many different functions are there from a set with 10 

elements to sets with the following numbers of elements? 

a) 2 b) 3 c) 4 d) 5 

How many one-to-one functions are there from a set with 

five elements to sets with the following number of ele- 

ments? 

a) 4 b) 5 c) 6 

How many functions are there from the set 

where n is a positive integer, to the set {0, 1 


d) 7 
{WeDo tt} 
}? 

How many functions are there from the set {1, 2, ..., n}, 

where n is a positive integer, to the set {0, 1} 

a) that are one-to-one? 

b) that assign 0 to both 1 and n? 

c) that assign 1 to exactly one of the positive integers 


less than n? 


How many partial functions (see Section 2.3) are there 
from a set with five elements to sets with each of these 
number of elements? 

a) 1 b) 2 c) 5 d) 9 

How many partial functions (see Definition 13 of Section 
2.3) are there from a set with m elements to a set with n 
elements, where m and n are positive integers? 

How many subsets of a set with 100 elements have more 
than one element? 

A palindrome is a string whose reversal is identical to 
the string. How many bit strings of length n are palin- 
dromes? 

How many 4-element DNA sequences 

a) do not contain the base T? 

b) contain the sequence ACG? 
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c) contain all four bases A, T, C, and G? 

d) contain exactly three of the four bases A, T, C, 
and G? 

How many 4-element RNA sequences 

a) contain the base U? 

b) do not contain the sequence CUG? 

c) do not contain all four bases A, U, C, and G? 

d) contain exactly two of the four bases A, U, C, and G? 


On each of the 22 work days in a particular month, every 
employee of a start-up venture was sent a company com- 
munication. If a total of 4642 total company communica- 
tions were sent, how many employees does the company 
have, assuming that no staffing changes were made that 
month? 


At a large university, 434 freshmen, 883 sophomores, 
and 43 juniors are enrolled in an introductory algorithms 
course. How many sections of this course need to be 
scheduled to accommodate all these students if each sec- 
tion contains 34 students? 


How many ways are there to seat four of a group of ten 
people around a circular table where two seatings are 
considered the same when everyone has the same imme- 
diate left and immediate right neighbor? 


How many ways are there to seat six people around a cir- 
cular table where two seatings are considered the same 
when everyone has the same two neighbors without re- 
gard to whether they are right or left neighbors? 


In how many ways can a photographer at a wedding ar- 

range 6 people in a row from a group of 10 people, where 

the bride and the groom are among these 10 people, if 

a) the bride must be in the picture? 

b) both the bride and groom must be in the picture? 

c) exactly one of the bride and the groom is in the pic- 
ture? 

In how many ways can a photographer at a wedding ar- 


range six people in a row, including the bride and groom, 
if 


a) the bride must be next to the groom? 

b) the bride is not next to the groom? 

c) the bride is positioned somewhere to the left of the 
groom? 

How many bit strings of length seven either begin with 

two Os or end with three 1s? 


How many bit strings of length 10 either begin with three 
Os or end with two 0s? 


How many bit strings of length 10 contain either five con- 
secutive Os or five consecutive 1s? 


How many bit strings of length eight contain either three 
consecutive Os or four consecutive 1s? 


Every student in a discrete mathematics class is either 
a computer science or a mathematics major or is a joint 
major in these two subjects. How many students are in 
the class if there are 38 computer science majors (includ- 
ing joint majors), 23 mathematics majors (including joint 
majors), and 7 joint majors? 
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How many positive integers not exceeding 100 are divis- 
ible either by 4 or by 6? 


How many different initials can someone have if a person 
has at least two, but no more than five, different initials? 
Assume that each initial is one of the 26 uppercase letters 
of the English language. 


Suppose that a password for a computer system must have 
at least 8, but no more than 12, characters, where each 
character in the password is a lowercase English letter, 
an uppercase English letter, a digit, or one of the six spe- 
cial characters *, >, <, !, +, and =. 


a) How many different passwords are available for this 
computer system? 


b) How many of these passwords contain at least one oc- 
currence of at least one of the six special characters? 


c) Using your answer to part (a), determine how long it 
takes a hacker to try every possible password, assum- 
ing that it takes one nanosecond for a hacker to check 
each possible password. 


The name of a variable in the C programming language is 
a string that can contain uppercase letters, lowercase let- 
ters, digits, or underscores. Further, the first character in 
the string must be a letter, either uppercase or lowercase, 
or an underscore. If the name of a variable is determined 
by its first eight characters, how many different variables 
can be named in C? (Note that the name of a variable may 
contain fewer than eight characters.) 


The name of a variable in the JAVA programming lan- 
guage is a string of between 1 and 65,535 characters, in- 
clusive, where each character can be an uppercase or a 
lowercase letter, a dollar sign, an underscore, or a digit, 
except that the first character must not be a digit. Deter- 
mine the number of different variable names in JAVA. 


The International Telecommunications Union (ITU) 
specifies that a telephone number must consist of a coun- 
try code with between 1 and 3 digits, except that the 
code 0 is not available for use as a country code, followed 
by a number with at most 15 digits. How many available 
possible telephone numbers are there that satisfy these 
restrictions? 


Suppose that at some future time every telephone in the 
world is assigned a number that contains a country code 
1 to 3 digits long, that is, of the form X, XX, or XXX, 
followed by a 10-digit telephone number of the form 
NXX-NXX-XXXX (as described in Example 8). How 
many different telephone numbers would be available 
worldwide under this numbering plan? 


A key in the Vigenére cryptosystem is a string of English 
letters, where the case of the letters does not matter. How 
many different keys for this cryptosystem are there with 
three, four, five, or six letters? 


A wired equivalent privacy (WEP) key for a wireless fi- 
delity (WiFi) network is a string of either 10, 26, or 58 
hexadecimal digits. How many different WEP keys are 
there? 
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6.1 The Basics of Counting 419 


Suppose that p and q are prime numbers and that n = pq. 
Use the principle of inclusion—exclusion to find the num- 
ber of positive integers not exceeding n that are relatively 
prime to n. 


Use the principle of inclusion—exclusion to find the num- 
ber of positive integers less than 1,000,000 that are not 
divisible by either 4 or by 6. 


Use a tree diagram to find the number of bit strings of 
length four with no three consecutive Os. 


How many ways are there to arrange the letters a, b, c, 
and d such that a is not followed immediately by b? 


Use a tree diagram to find the number of ways that the 
World Series can occur, where the first team that wins 
four games out of seven wins the series. 


Use a tree diagram to determine the number of subsets 
of {3, 7,9, 11, 24} with the property that the sum of the 
elements in the subset is less than 28. 


a) Suppose that a store sells six varieties of soft drinks: 
cola, ginger ale, orange, root beer, lemonade, and 
cream soda. Use a tree diagram to determine the num- 
ber of different types of bottles the store must stock to 
have all varieties available in all size bottles if all vari- 
eties are available in 12-ounce bottles, all but lemon- 
ade are available in 20-ounce bottles, only cola and 
ginger ale are available in 32-ounce bottles, and all 
but lemonade and cream soda are available in 64- 
ounce bottles? 


b) Answer the question in part (a) using counting rules. 


a) Suppose that a popular style of running shoe is avail- 
able for both men and women. The woman’s shoe 
comes in sizes 6, 7, 8, and 9, and the man’s shoe 
comes in sizes 8, 9, 10, 11, and 12. The man’s 
shoe comes in white and black, while the woman’s 
shoe comes in white, red, and black. Use a tree dia- 
gram to determine the number of different shoes that 
a store has to stock to have at least one pair of this 
type of running shoe for all available sizes and colors 
for both men and women. 


b) Answer the question in part (a) using counting rules. 


Determine the number of matches played in a single- 
elimination tournament with n players, where for each 
game between two players the winner goes on, but the 
loser is eliminated. 


Determine the minimum and the maximum number of 
matches that can be played in a double-elimination tour- 
nament with n players, where after each game between 
two players, the winner goes on and the loser goes on if 
and only if this is not a second loss. 


Use the product rule to show that there are 2?" different 
truth tables for propositions in n variables. 


Use mathematical induction to prove the sum rule for m 
tasks from the sum rule for two tasks. 


Use mathematical induction to prove the product rule 
for m tasks from the product rule for two tasks. 
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77. How many diagonals does a convex polygon with n sides 
have? (Recall that a polygon is convex if every line seg- 
ment connecting two points in the interior or boundary of 
the polygon lies entirely within this set and that a diago- 
nal of a polygon is a line segment connecting two vertices 
that are not adjacent.) 


78. Data are transmitted over the Internet in datagrams, 
which are structured blocks of bits. Each datagram con- 
tains header information organized into a maximum of 14 
different fields (specifying many things, including the 
source and destination addresses) and a data area that 
contains the actual data that are transmitted. One of the 
14 header fields is the header length field (denoted by 
HLEN), which is specified by the protocol to be 4 bits 
long and that specifies the header length in terms of 32- 
bit blocks of bits. For example, if HLEN = 0110, the 
header is made up of six 32-bit blocks. Another of the 14 
header fields is the 16-bit-long total length field (denoted 


The Pigeonhole Principle 


6.2.1 Introduction 


by TOTAL LENGTH), which specifies the length in bits 

of the entire datagram, including both the header fields 

and the data area. The length of the data area is the total 
length of the datagram minus the length of the header. 

a) The largest possible value of TOTAL LENGTH 
(which is 16 bits long) determines the maximum 
total length in octets (blocks of 8 bits) of an Internet 
datagram. What is this value? 

b) The largest possible value of HLEN (which is 4 bits 
long) determines the maximum total header length in 
32-bit blocks. What is this value? What is the maxi- 
mum total header length in octets? 

c) The minimum (and most common) header length is 
20 octets. What is the maximum total length in octets 
of the data area of an Internet datagram? 

d) How many different strings of octets in the data area 
can be transmitted if the header length is 20 octets 
and the total length is as long as possible? 


, Suppose that a flock of 20 pigeons flies into a set of 19 pigeonholes to roost. Because there are 
HINKS » 20 pigeons but only 19 pigeonholes, a least one of these 19 pigeonholes must have at least two 
pigeons in it. To see why this is true, note that if each pigeonhole had at most one pigeon in it, 
at most 19 pigeons, one per hole, could be accommodated. This illustrates a general principle 
called the pigeonhole principle, which states that if there are more pigeons than pigeonholes, 
then there must be at least one pigeonhole with at least two pigeons in it (see Figure 1). This 

principle is extremely useful; it applies to much more than pigeons and pigeonholes. 


THEOREM1 THE PIGEONHOLE PRINCIPLE If k is a positive integer and k + 1 or more objects 
are placed into k boxes, then there is at least one box containing two or more of the objects. 
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FIGURE 1 There are more pigeons than pigeonholes. 


(c) 
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Proof: We prove the pigeonhole principle using a proof by contraposition. Suppose that none of 
the k boxes contains more than one object. Then the total number of objects would be at most k. 
This is a contradiction, because there are at least k + 1 objects. < 


The pigeonhole principle is also called the Dirichlet drawer principle, after the nineteenth- 
century German mathematician G. Lejeune Dirichlet, who often used this principle in his work. 
(Dirichlet was not the first person to use this principle; a demonstration that there were at least 
two Parisians with the same number of hairs on their heads dates back to the 17th century— 
see Exercise 35.) It is an important additional proof technique supplementing those we have 
developed in earlier chapters. We introduce it in this chapter because of its many important 
applications to combinatorics. 

We will illustrate the usefulness of the pigeonhole principle. We first show that it can be 
used to prove a useful corollary about functions. 


A function f from a set with k + 1 or more elements to a set with k elements is not one-to-one. 


Proof: Suppose that for each element y in the codomain of f we have a box that contains all 
elements x of the domain of f such that f(x) = y. Because the domain contains k + 1 or more 
elements and the codomain contains only k elements, the pigeonhole principle tells us that one 
of these boxes contains two or more elements x of the domain. This means that f cannot be 
one-to-one. < 


Examples 1-3 show how the pigeonhole principle is used. 


Among any group of 367 people, there must be at least two with the same birthday, because 
there are only 366 possible birthdays. < 


In any group of 27 English words, there must be at least two that begin with the same letter, 
because there are 26 letters in the English alphabet. < 


How many students must be in a class to guarantee that at least two students receive the same 
score on the final exam, if the exam is graded on a scale from 0 to 100 points? 


Solution: There are 101 possible scores on the final. The pigeonhole principle shows that among 
any 102 students there must be at least 2 students with the same score. < 


G. LEJEUNE DIRICHLET (1805-1859) G. Lejeune Dirichlet was born into a Belgian family living near 
Cologne, Germany. His father was a postmaster. He became passionate about mathematics at a young age. 
He was spending all his spare money on mathematics books by the time he entered secondary school in 
Bonn at the age of 12. At 14 he entered the Jesuit College in Cologne, and at 16 he began his studies at 
the University of Paris. In 1825 he returned to Germany and was appointed to a position at the Univer- 
sity of Breslau. In 1828 he moved to the University of Berlin. In 1855 he was chosen to succeed Gauss at 
the University of Göttingen. Dirichlet is said to be the first person to master Gauss’s Disquisitiones Arith- 
meticae, which appeared 20 years earlier. He is said to have kept a copy at his side even when he trav- 
eled. Dirichlet made many important discoveries in number theory, including the theorem that there are in- 
finitely many primes in arithmetical progressions an + b when a and b are relatively prime. He proved the 
n = 5 case of Fermat’s last theorem, that there are no nontrivial solutions in integers to » + y3 = z. Dirichlet 


also made many contributions to analysis. Dirichlet was considered to be an excellent teacher who could explain ideas with great 
clarity. He was married to Rebecka Mendelssohn, one of the sisters of the composer Felix Mendelssohn. 
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The pigeonhole principle is a useful tool in many proofs, including proofs of surprising 
results, such as that given in Example 4. 


Show that for every integer n there is a multiple of n that has only Os and Is in its decimal 
expansion. 


Solution: Let n be a positive integer. Consider the n + 1 integers 1, 11, 111,..., 11... 1 (where 
the last integer in this list is the integer with n + 1 1s in its decimal expansion). Note that there 
are n possible remainders when an integer is divided by n. Because there are n + 1 integers in 
this list, by the pigeonhole principle there must be two with the same remainder when divided 
by n. The larger of these integers less the smaller one is a multiple of n, which has a decimal 
expansion consisting entirely of Os and 1s. < 


6.2.2 The Generalized Pigeonhole Principle 


The pigeonhole principle states that there must be at least two objects in the same box when 
there are more objects than boxes. However, even more can be said when the number of objects 
exceeds a multiple of the number of boxes. For instance, among any set of 21 decimal digits 
there must be 3 that are the same. This follows because when 21 objects are distributed into 
10 boxes, one box must have more than 2 objects. 


THE GENERALIZED PIGEONHOLE PRINCIPLE If N objects are placed into k 
boxes, then there is at least one box containing at least [N/k] objects. 


Proof: We will use a proof by contraposition. Suppose that none of the boxes contains more 
than [N/k] — 1 objects. Then, the total number of objects is at most 


N N 
k([>]-1) <k((S41)-1) =n, 
k k 
where the inequality [N/k] < (NV/k) + 1 has been used. Thus, the total number of objects is 
less than N. This completes the proof by contraposition. < 


A common type of problem asks for the minimum number of objects such that at least r 
of these objects must be in one of k boxes when these objects are distributed among the boxes. 
When we have N objects, the generalized pigeonhole principle tells us there must be at least 
r objects in one of the boxes as long as [N/k] > r. The smallest integer N with N/k >r-1, 
namely, N = k(r — 1) + 1, is the smallest integer satisfying the inequality [N/k] > r. Could a 
smaller value of N suffice? The answer is no, because if we had k(r — 1) objects, we could put 
r — 1 of them in each of the k boxes and no box would have at least r objects. 

When thinking about problems of this type, it is useful to consider how you can avoid having 
at least r objects in one of the boxes as you add successive objects. To avoid adding a rth object 
to any box, you eventually end up with r — 1 objects in each box. There is no way to add the 
next object without putting an rth object in that box. 

Examples 5-8 illustrate how the generalized pigeonhole principle is applied. 


Among 100 people there are at least [100/12] = 9 who were born in the same month. < 


What is the minimum number of students required in a discrete mathematics class to be sure 
that at least six will receive the same grade, if there are five possible grades, A, B, C, D, and F? 


Extra » 
Examples 


EXAMPLE 7 


A standard deck of 52 
cards has 13 kinds of 
cards, with four cards of 
each of kind, one in 
each of the four suits, 
hearts, diamonds, 
spades, and clubs. 


EXAMPLE 8 


EXAMPLE 9 
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Solution: The minimum number of students needed to ensure that at least six students receive 
the same grade is the smallest integer N such that [N/5] = 6. The smallest such integer is 
N=5-5+1= 26. If you have only 25 students, it is possible for there to be five who have 
received each grade so that no six students have received the same grade. Thus, 26 is the 
minimum number of students needed to ensure that at least six students will receive the same 
grade. < 


a) How many cards must be selected from a standard deck of 52 cards to guarantee that at least 
three cards of the same suit are selected? 

b) How many must be selected from a standard deck of 52 cards to guarantee that at least three 
hearts are selected? 


Solution: a) Suppose there are four boxes, one for each suit, and as cards are selected they are 
placed in the box reserved for cards of that suit. Using the generalized pigeonhole principle, 
we see that if N cards are selected, there is at least one box containing at least [N/4] cards. 
Consequently, we know that at least three cards of one suit are selected if [N/4] > 3. The 
smallest integer N such that [V/4] > 3 is N=2-4+1=9, so nine cards suffice. Note that if 
eight cards are selected, it is possible to have two cards of each suit, so more than eight cards are 
needed. Consequently, nine cards must be selected to guarantee that at least three cards of one 
suit are chosen. One good way to think about this is to note that after the eighth card is chosen, 
there is no way to avoid having a third card of some suit. 


b) We do not use the generalized pigeonhole principle to answer this question, because we want 
to make sure that there are three hearts, not just three cards of one suit. Note that in the worst 
case, we can select all the clubs, diamonds, and spades, 39 cards in all, before we select a single 
heart. The next three cards will be all hearts, so we may need to select 42 cards to get three 
hearts. < 


What is the least number of area codes needed to guarantee that the 25 million phones in a state 
can be assigned distinct 10-digit telephone numbers? (Assume that telephone numbers are of 
the form NXX-NXX-XXXX, where the first three digits form the area code, N represents a digit 
from 2 to 9 inclusive, and X represents any digit.) 


Solution: There are eight million different phone numbers of the form NXX-XXXX (as shown in 
Example 8 of Section 6.1). Hence, by the generalized pigeonhole principle, among 25 million 
telephones, at least [25,000,000/8,000,000] = 4 of them must have identical phone numbers. 
Hence, at least four area codes are required to ensure that all 10-digit numbers are different. <4 


Example 9, although not an application of the generalized pigeonhole principle, makes use 
of similar principles. 


Suppose that a computer science laboratory has 15 workstations and 10 servers. A cable can be 
used to directly connect a workstation to a server. For each server, only one direct connection to 
that server can be active at any time. We want to guarantee that at any time any set of 10 or fewer 
workstations can simultaneously access different servers via direct connections. Although we 
could do this by connecting every workstation directly to every server (using 150 connections), 
what is the minimum number of direct connections needed to achieve this goal? 


Solution: Suppose that we label the workstations W,, W3, ..., W,; and the servers S,, S5, ..., S10- 
First, we would like to find a way for there to be far fewer than 150 direct connections between 
workstations and servers to achieve our goal. One promising approach is to directly connect 
W, to S, for k = 1,2, ..., 10 and then to connect each of W,,, Wiz, Wiz, Wig, and Wj; to all 


424 6/ Counting 


EXAMPLE 10 


EXAMPLE 11 


10 servers. This gives us a total of 10 + 5 - 10 = 60 direct connections. We need to determine 
whether with this configuration any set of 10 or fewer workstations can simultaneously access 
different servers. We note that if workstation W is included with 1 < j < 10, it can access server 
Si, and for each workstation W, with k > 11 included, there must be a corresponding workstation 
W with 1 <j < 10 not included, so W, can access server Sj. (This follows because there are at 
least as many available servers Si as there are workstations W with 1 < j < 10 not included.) So, 
any set of 10 or fewer workstations are able to simultaneously access different servers. 

But can we use fewer than 60 direct connections? Suppose there are fewer than 60 direct 
connections between workstations and servers. Then some server would be connected to at 
most [59/10] = 5 workstations. (If all servers were connected to at least six workstations, there 
would be at least 6 - 10 = 60 direct connections.) This means that the remaining nine servers 
are not enough for the other 10 or more workstations to simultaneously access different servers. 
Consequently, at least 60 direct connections are needed. It follows that 60 is the answer. < 


6.2.3 Some Elegant Applications of the Pigeonhole Principle 


In many interesting applications of the pigeonhole principle, the objects to be placed in boxes 
must be chosen in a clever way. A few such applications will be described here. 


During a month with 30 days, a baseball team plays at least one game a day, but no more 
than 45 games. Show that there must be a period of some number of consecutive days during 
which the team must play exactly 14 games. 


Solution: Let a; be the number of games played on or before the jth day of the month. Then 
1, Ay, ..., day is an increasing sequence of distinct positive integers, with 1 < a; S 45. More- 
over, a; + 14, a, + 14, ..., a39 + 14 is also an increasing sequence of distinct positive integers, 
with 15 < a, + 14 < 59. 

The 60 positive integers a), a, ..., 39, 4; + 14, a, + 14, ..., a39 + 14 are all less than or 
equal to 59. Hence, by the pigeonhole principle two of these integers are equal. Because the 
integers ajj = ], 2, ..., 30 are all distinct and the integers a;+ 14,7 = 1, 2, ..., 30 are all distinct, 
there must be indices i and j with a, = a; + 14. This means that exactly 14 games were played 
from day j + 1 to day i. 


Show that among any n + 1 positive integers not exceeding 2n there must be an integer that 
divides one of the other integers. 


Solution: Write each of the n + 1 integers a), do, ..., 4,4; as a power of 2 times an odd integer. 
In other words, let a= 24g, for j = 1,2,...,n+ 1, where k; is a nonnegative integer and qj is 
odd. The integers q4, q2 ..., 4,4, are all odd positive integers less than 2n. Because there are 
only n odd positive integers less than 2n, it follows from the pigeonhole principle that two of 
the integers q4, q2, .--, Jn4 Must be equal. Therefore, there are distinct integers i and j such that 
qi = q. Let q be the common value of q; and q;. Then, a; = 2kq and q = 2iq. It follows that if 
k; < k, then a; divides a; while if k; > &, then q divides a;. < 


A clever application of the pigeonhole principle shows the existence of an increasing or a 
decreasing subsequence of a certain length in a sequence of distinct integers. We review some 
definitions before this application is presented. Suppose that a,, a», ..., dy is a Sequence of real 
numbers. A subsequence of this sequence is a sequence of the form Aj» Qis -s Aj where 1 < 
i <i < +++ <i,, < N. Hence, a subsequence is a sequence obtained from the original sequence 


by including some of the terms of the original sequence in their original order, and perhaps not 
including other terms. A sequence is called strictly increasing if each term is larger than the 
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one that precedes it, and it is called strictly decreasing if each term is smaller than the one that 
precedes it. 


Every sequence of n? + 1 distinct real numbers contains a subsequence of length n + 1 that 
is either strictly increasing or strictly decreasing. 


We give an example before presenting the proof of Theorem 3. 


The sequence 8, 11, 9, 1, 4, 6, 12, 10, 5, 7 contains 10 terms. Note that 10 = 32 + 1. There 
are four strictly increasing subsequences of length four, namely, 1, 4, 6, 12; 1, 4, 6, 7; 1, 
4, 6, 10; and 1, 4, 5, 7. There is also a strictly decreasing subsequence of length four, 
namely, 11, 9, 6, 5. < 


The proof of the theorem will now be given. 


Proof: Let ay, dy, ... , 4,24, be a sequence of n? + 1 distinct real numbers. Associate an ordered 
pair with each term of the sequence, namely, associate (i,, d,) to the term a,, where i, is the 
length of the longest increasing subsequence starting at a,, and d, is the length of the longest 
decreasing subsequence starting at a,. 

Suppose that there are no increasing or decreasing subsequences of length n+ 1. Then 
i, and d, are both positive integers less than or equal to n, for k = 1, 2, ..., n? +1. Hence, 
by the product rule there are n? possible ordered pairs for (i,, d,). By the pigeonhole prin- 
ciple, two of these n? + 1 ordered pairs are equal. In other words, there exist terms a, and 
d, with s <t such that i, =i, and d, =d,. We will show that this is impossible. Because 
the terms of the sequence are distinct, either a, < a, or a, > a, If a, < a, then, because 
i, = i, an increasing subsequence of length i,+ 1 can be built starting at a,, by taking a, 
followed by an increasing subsequence of length i, beginning at a,. This is a contradiction. 
Similarly, if a, > a,, the same reasoning shows that d, must be greater than d,, which is a 
contradiction. < 


The final example shows how the generalized pigeonhole principle can be applied to an im- 
portant part of combinatorics called Ramsey theory, after the English mathematician F. P. Ram- 
sey. In general, Ramsey theory deals with the distribution of subsets of elements of sets. 


Assume that in a group of six people, each pair of individuals consists of two friends or two 
enemies. Show that there are either three mutual friends or three mutual enemies in the group. 


Solution: Let A be one of the six people. Of the five other people in the group, there are either 
three or more who are friends of A, or three or more who are enemies of A. This follows from 


FRANK PLUMPTON RAMSEY (1903-1930) Frank Plumpton Ramsey, son of the president of Magdalene 
College, Cambridge, was educated at Winchester and Trinity Colleges. After graduating in 1923, he was elected 
a fellow of King’s College, Cambridge, where he spent the remainder of his life. Ramsey made important 
contributions to mathematical logic. What we now call Ramsey theory began with his clever combinatorial 
arguments, published in the paper “On a Problem of Formal Logic.” Ramsey also made contributions to the 
mathematical theory of economics. He was noted as an excellent lecturer on the foundations of mathematics. 
According to one of his brothers, he was interested in almost everything, including English literature and politics. 
Ramsey was married and had two daughters. His death at the age of 26 resulting from chronic liver problems 
deprived the mathematical community and Cambridge University of a brilliant young scholar. 
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Exercises 


the generalized pigeonhole principle, because when five objects are divided into two sets, one 
of the sets has at least [5/2] = 3 elements. In the former case, suppose that B, C, and D are 
friends of A. If any two of these three individuals are friends, then these two and A form a group 
of three mutual friends. Otherwise, B, C, and D form a set of three mutual enemies. The proof 
in the latter case, when there are three or more enemies of A, proceeds in a similar manner. <4 


The Ramsey number R(m, n), where m andn are positive integers greater than or equal to 2, 
denotes the minimum number of people at a party such that there are either m mutual friends 
or n mutual enemies, assuming that every pair of people at the party are friends or enemies. 
Example 13 shows that R(3, 3) < 6. We conclude that R(3, 3) = 6 because in a group of five 
people where every two people are friends or enemies, there may not be three mutual friends or 
three mutual enemies (see Exercise 28). 

It is possible to prove some useful properties about Ramsey numbers, but for the most 
part it is difficult to find their exact values. Note that by symmetry it can be shown that 
R(m, n) = R(n, m) (see Exercise 32). We also have R(2, n) = n for every positive integer n > 2 
(see Exercise 31). The exact values of only nine Ramsey numbers R(m, n) with 3 < m < n are 
known, including R(4, 4) = 18. Only bounds are known for many other Ramsey numbers, in- 
cluding R(5, 5), which is known to satisfy 43 < R(5, 5) < 49. The reader interested in learning 
more about Ramsey numbers should consult [MiRo91] or [GrRoSp90]. 


. Show that in any set of six classes, each meeting regu- 
larly once a week on a particular day of the week, there 
must be two that meet on the same day, assuming that no 
classes are held on weekends. 


. Show that if there are 30 students in a class, then at least 

two have last names that begin with the same letter. 

. A drawer contains a dozen brown socks and a dozen black 

socks, all unmatched. A man takes socks out at random 

in the dark. 

a) How many socks must he take out to be sure that he 
has at least two socks of the same color? 

b) How many socks must he take out to be sure that he 
has at least two black socks? 

. A bowl contains 10 red balls and 10 blue balls. A woman 

selects balls at random without looking at them. 

a) How many balls must she select to be sure of having 
at least three balls of the same color? 

b) How many balls must she select to be sure of having 


10. 


11. 


that there are two students with the same professor who 
earned the same final examination score? 


. Show that among any group of five (not necessarily con- 


secutive) integers, there are two with the same remainder 
when divided by 4. 


. Let d be a positive integer. Show that among any group of 


d + 1 (not necessarily consecutive) integers there are two 
with exactly the same remainder when they are divided 
by d. 


. Let n be a positive integer. Show that in any set of n con- 


secutive integers there is exactly one divisible by n. 
Show that if f is a function from S to T, where S and T 
are finite sets with |S] > |T], then there are elements s, 
and s, in S such that f(s,) = f (s2), or in other words, f is 
not one-to-one. 

What is the minimum number of students, each of whom 
comes from one of the 50 states, who must be enrolled in 
a university to guarantee that there are at least 100 who 
come from the same state? 


at least three blue balls? *12. Let (x, y;), i = 1, 2,3, 4,5, be a set of five distinct points 
. Undergraduate students at a college belong to one of four with integer coordinates in the xy plane. Show that the 
groups depending on the year in which they are expected midpoint of the line joining at least one pair of these 
to graduate. Each student must choose one of 21 differ- points has integer coordinates. 
ent majors. How many students are needed to assure that *13. Let (x,y,z). i= 1, 2, 3, 4, 5, 6,7, 8,9, be a set of nine 
there are two students expected to graduate in the same distinct points with integer coordinates in xyz space. 
year who have the same major? Show that the midpoint of at least one pair of these points 
. There are six professors teaching the introductory dis- has integer coordinates. 
crete mathematics class at a university. The same final 14. How many ordered pairs of integers (a,b) are 


exam is given by all six professors. If the lowest possi- 
ble score on the final is 0 and the highest possible score 
is 100, how many students must there be to guarantee 


needed to guarantee that there are two ordered pairs 
(a,,b,) and (ay, b,) such that a, mod 5 =a, mod 5 
and b, mod 5 = b, mod 5? 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


* 25, 


*%* 26, 


a) Show that if five integers are selected from the first 
eight positive integers, there must be a pair of these 
integers with a sum equal to 9. 

b) Is the conclusion in part (a) true if four integers are 
selected rather than five? 


a) Show that if seven integers are selected from the first 
10 positive integers, there must be at least two pairs 
of these integers with the sum 11. 

b) Is the conclusion in part (a) true if six integers are 
selected rather than seven? 


How many numbers must be selected from the set 
{1, 2, 3, 4, 5, 6} to guarantee that at least one pair of these 
numbers add up to 7? 


How many numbers must be selected from the set 
{1, 3,5, 7, 9, 11, 13, 15} to guarantee that at least one pair 
of these numbers add up to 16? 


A company stores products in a warehouse. Storage bins 
in this warehouse are specified by their aisle, location in 
the aisle, and shelf. There are 50 aisles, 85 horizontal lo- 
cations in each aisle, and 5 shelves throughout the ware- 
house. What is the least number of products the company 
can have so that at least two products must be stored in 
the same bin? 


Suppose that there are nine students in a discrete mathe- 

matics class at a small college. 

a) Show that the class must have at least five male stu- 
dents or at least five female students. 

b) Show that the class must have at least three male stu- 
dents or at least seven female students. 

Suppose that every student in a discrete mathematics 

class of 25 students is a freshman, a sophomore, or a 

junior. 

a) Show that there are at least nine freshmen, at least 
nine sophomores, or at least nine juniors in the class. 

b) Show that there are either at least three freshmen, at 
least 19 sophomores, or at least five juniors in the 
class. 

Find an increasing subsequence of maximal length and 

a decreasing subsequence of maximal length in the se- 

quence 22, 5, 7, 2, 23, 10, 15, 21, 3, 17. 


Construct a sequence of 16 positive integers that has no 
increasing or decreasing subsequence of five terms. 


Show that if there are 101 people of different heights 
standing in a line, it is possible to find 11 people in the 
order they are standing in the line with heights that are 
either increasing or decreasing. 


Show that whenever 25 girls and 25 boys are seated 
around a circular table there is always a person both of 
whose neighbors are boys. 


Suppose that 21 girls and 21 boys enter a mathemat- 
ics competition. Furthermore, suppose that each entrant 
solves at most six questions, and for every boy-girl pair, 
there is at least one question that they both solved. Show 
that there is a question that was solved by at least three 
girls and at least three boys. 


* 27, 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


37. 


38. 


6.2 The Pigeonhole Principle 427 


Describe an algorithm in pseudocode for producing the 
largest increasing or decreasing subsequence of a se- 
quence of distinct integers. 


Show that in a group of five people (where any two people 
are either friends or enemies), there are not necessarily 
three mutual friends or three mutual enemies. 


Show that in a group of 10 people (where any two people 
are either friends or enemies), there are either three mu- 
tual friends or four mutual enemies, and there are either 
three mutual enemies or four mutual friends. 


Use Exercise 29 to show that among any group of 20 peo- 
ple (where any two people are either friends or enemies), 
there are either four mutual friends or four mutual ene- 
mies. 


Show that if n is an integer with n > 2, then the Ramsey 
number R(2, n) equals n. (Recall that Ramsey numbers 
were discussed after Example 13 in Section 6.2.) 


Show that if m and n are integers with m > 2 and n > 2, 
then the Ramsey numbers R(m, n) and R(n, m) are equal. 
(Recall that Ramsey numbers were discussed after Ex- 
ample 13 in Section 6.2.) 


Show that there are at least six people in California (pop- 
ulation: 39 million) with the same three initials who were 
born on the same day of the year (but not necessarily in 
the same year). Assume that everyone has three initials. 


Show that if there are 100,000,000 wage earners in the 
United States who earn less than 1,000,000 dollars (but 
at least a penny), then there are two who earned exactly 
the same amount of money, to the penny, last year. 


In the 17th century, there were more than 800,000 inhabi- 
tants of Paris. At the time, it was believed that no one had 
more than 200,000 hairs on their head. Assuming these 
numbers are correct and that everyone has at least one 
hair on their head (that is, no one is completely bald), use 
the pigeonhole principle to show, as the French writer 
Pierre Nicole did, that there had to be two Parisians with 
the same number of hairs on their heads. Then use the 
generalized pigeonhole principle to show that there had 
to be at least five Parisians at that time with the same 
number of hairs on their heads. 


Assuming that no one has more than 1,000,000 hairs on 
their head and that the population of New York City was 
8,537,673 in 2016, show there had to be at least nine peo- 
ple in New York City in 2016 with the same number of 
hairs on their heads. 


There are 38 different time periods during which classes 
at a university can be scheduled. If there are 677 different 
classes, how many different rooms will be needed? 


A computer network consists of six computers. Each 
computer is directly connected to at least one of the other 
computers. Show that there are at least two computers in 
the network that are directly connected to the same num- 
ber of other computers. 
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39. 


40. 
41. 
* 42. 


43. 


* 44, 


45. 


Permutations and Combinations 


A computer network consists of six computers. Each 
computer is directly connected to zero or more of the 
other computers. Show that there are at least two com- 
puters in the network that are directly connected to the 
same number of other computers. [Hint: It is impossible 
to have a computer linked to none of the others and a 
computer linked to all the others.] 

Find the least number of cables required to connect eight 
computers to four printers to guarantee that for every 
choice of four of the eight computers, these four com- 
puters can directly access four different printers. Justify 
your answer. 

Find the least number of cables required to connect 100 
computers to 20 printers to guarantee that every subset 
of 20 computers can directly access 20 different printers. 
(Here, the assumptions about cables and computers are 
the same as in Example 9.) Justify your answer. 

Prove that at a party where there are at least two people, 
there are two people who know the same number of other 
people there. 

An arm wrestler is the champion for a period of 75 hours. 
(Here, by an hour, we mean a period starting from an ex- 
act hour, such as 1 P.M., until the next hour.) The arm 
wrestler had at least one match an hour, but no more than 
125 total matches. Show that there is a period of consec- 
utive hours during which the arm wrestler had exactly 24 
matches. 

Is the statement in Exercise 43 true if 24 is replaced by 

a) 2? b) 23? c) 25? d) 30? 
Show that if f is a function from S to T, where S and T are 
nonempty finite sets and m = [|S|/|T|], then there are at 


46. 


* 47, 


48. 


* 49, 


least m elements of S mapped to the same value of T. That 
is, show that there are distinct elements s,, 59, ..., Sm of S 


such that f (s1) = f(s.) = + =f(5,,). 


There are 51 houses on a street. Each house has an ad- 
dress between 1000 and 1099, inclusive. Show that at 
least two houses have addresses that are consecutive in- 
tegers. 


Let x be an irrational number. Show that for some positive 
integer j not exceeding the positive integer n, the absolute 
value of the difference between jx and the nearest integer 
to jx is less than I/n. 


Let nj, np... n, be positive integers. Show that if 
nyi +m +++ +n,—t+ 1 objects are placed into t boxes, 
then for some i, i = 1, 2, ..., t, the ith box contains at least 
n; objects. 


An alternative proof of Theorem 3 based on the general- 

ized pigeonhole principle is outlined in this exercise. The 

notation used is the same as that used in the proof in the 
text. 

a) Assume that i, <n for k = 1, 2, ..., n? +1. Use the 
generalized pigeonhole principle to show that there 
are n + ltermsa,,d,,,...,d, , With i =i, = = 
ix,» Where 1 < k < ka < ++ < kay. 

b) Show that ar, > A, for j= 1,2,...,n. [Hint: As- 
sume that ay, < ak > and show that this implies that 
ik, > ik , which is a contradiction. ] 

c) Use parts (a) and (b) to show that if there is no increas- 


ing subsequence of length n + 1, then there must be a 
decreasing subsequence of this length. 


6.3.1 Introduction 


Many counting problems can be solved by finding the number of ways to arrange a specified 
number of distinct elements of a set of a particular size, where the order of these elements 
matters. Many other counting problems can be solved by finding the number of ways to select 
a particular number of elements from a set of a particular size, where the order of the elements 
selected does not matter. For example, in how many ways can we select three students from a 
group of five students to stand in line for a picture? How many different committees of three 
students can be formed from a group of four students? In this section we will develop methods 


to answer questions such as these. 


6.3.2 Permutations 


We begin by solving the first question posed in the introduction to this section, as well as related 


questions. 


EXAMPLE 1 


In how many ways can we select three students from a group of five students to stand in line for 


a picture? In how many ways can we arrange all five of these students in a line for a picture? 


Extra » 
Examples 


Links » 


EXAMPLE 2 


EXAMPLE 3 


THEOREM 1 
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Solution: First, note that the order in which we select the students matters. There are five ways 
to select the first student to stand at the start of the line. Once this student has been selected, 
there are four ways to select the second student in the line. After the first and second students 
have been selected, there are three ways to select the third student in the line. By the product 
rule, there are 5 - 4- 3 = 60 ways to select three students from a group of five students to stand 
in line for a picture. 

To arrange all five students in a line for a picture, we select the first student in five ways, 
the second in four ways, the third in three ways, the fourth in two ways, and the fifth in one 
way. Consequently, there are 5 -4-3-2-1 = 120 ways to arrange all five students in a line for 
a picture. < 


Example 1 illustrates how ordered arrangements of distinct objects can be counted. This leads 
to some terminology. 

A permutation of a set of distinct objects is an ordered arrangement of these objects. We 
also are interested in ordered arrangements of some of the elements of a set. An ordered ar- 
rangement of r elements of a set is called an r-permutation. 


Let S$ = {1, 2,3}. The ordered arrangement 3, 1, 2 is a permutation of S. The ordered arrange- 
ment 3, 2 is a 2-permutation of S. < 


The number of r-permutations of a set with n elements is denoted by P(n, r). We can find P(n, r) 
using the product rule. 


Let S = {a, b, c}. The 2-permutations of S are the ordered arrangements a, b; a, c; b, a; b, c; c, a; 
and c, b. Consequently, there are six 2-permutations of this set with three elements. There are 
always six 2-permutations of a set with three elements. There are three ways to choose the first 
element of the arrangement. There are two ways to choose the second element of the arrange- 
ment, because it must be different from the first element. Hence, by the product rule, we see that 
P(3, 2) = 3 - 2 = 6. the first element. By the product rule, it follows that P(3,2) =3-2=6. <4 


We now use the product rule to find a formula for P(n, r) whenever n and r are positive integers 
with | <r<n. 


If n is a positive integer and r is an integer with 1 < r < n, then there are 
P(n,r) = n(n - In -2)+-—r+)) 


r-permutations of a set with n distinct elements. 


Proof: We will use the product rule to prove that this formula is correct. The first element of the 
permutation can be chosen in n ways because there are n elements in the set. There are n — 1 
ways to choose the second element of the permutation, because there are n — 1 elements left 
in the set after using the element picked for the first position. Similarly, there are n — 2 ways 
to choose the third element, and so on, until there are exactly n — (r — 1) =n—r-+1 ways to 
choose the rth element. Consequently, by the product rule, there are 


n(n — 1)\(n—2)-(n-rt+1) 


r-permutations of the set. < 
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COROLLARY 1 


EXAMPLE 4 


EXAMPLE 5 


EXAMPLE 6 


Note that P(n, 0) = 1 whenever n is a nonnegative integer because there is exactly one way to 
order zero elements. That is, there is exactly one list with no elements in it, namely the empty list. 
We now state a useful corollary of Theorem 1. 


n! 


(n—r)! 


If n and r are integers with 0 < r < n, then P(n, r) = 


Proof: When n andr are integers with 1 < r < n, by Theorem 1 we have 


! 
P(n, r) =n(n— I(n— 2) (n—-r+ 1) = ——— 
(n—1r)! 
! 
Because ci = = = ] whenever n is a nonnegative integer, we see that the formula 
n—-O)! n! 
! 
P(n,r) = M’ also holds when r = 0. < 


(n-r)! 


By Theorem 1 we know that if n is a positive integer, then P(n, n) = n!. We will illustrate 
this result with some examples. 


How many ways are there to select a first-prize winner, a second-prize winner, and a third-prize 
winner from 100 different people who have entered a contest? 


Solution: Because it matters which person wins which prize, the number of ways to pick the 
three prize winners is the number of ordered selections of three elements from a set of 100 
elements, that is, the number of 3-permutations of a set of 100 elements. Consequently, the 
answer is 


P(100, 3) = 100 - 99 - 98 = 970,200. < 


Suppose that there are eight runners in a race. The winner receives a gold medal, the second- 
place finisher receives a silver medal, and the third-place finisher receives a bronze medal. How 
many different ways are there to award these medals, if all possible outcomes of the race can 
occur and there are no ties? 


Solution: The number of different ways to award the medals is the number of 3-permutations of 
a set with eight elements. Hence, there are P(8, 3) = 8 - 7 - 6 = 336 possible ways to award the 
medals. < 


Suppose that a saleswoman has to visit eight different cities. She must begin her trip in a spec- 
ified city, but she can visit the other seven cities in any order she wishes. How many possible 
orders can the saleswoman use when visiting these cities? 


Solution: The number of possible paths between the cities is the number of permutations of 
seven elements, because the first city is determined, but the remaining seven can be ordered 
arbitrarily. Consequently, there are 7! = 7-6-5-4-3-2-1=5040 ways for the saleswoman 
to choose her tour. If, for instance, the saleswoman wishes to find the path between the cities 
with minimum distance, and she computes the total distance for each possible path, she must 
consider a total of 5040 paths! < 


EXAMPLE 7 


EXAMPLE 8 


Links > 


EXAMPLE 9 


EXAMPLE 10 


THEOREM 2 
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How many permutations of the letters ABCDEFGH contain the string ABC ? 


Solution: Because the letters ABC must occur as a block, we can find the answer by finding the 
number of permutations of six objects, namely, the block ABC and the individual letters D, E, 
F, G, and H. Because these six objects can occur in any order, there are 6! = 720 permutations 
of the letters ABCDEFGH in which ABC occurs as a block. < 


6.3.3 Combinations 


We now turn our attention to counting unordered selections of objects. We begin by solving a 
question posed in the introduction to this section of the chapter. 


How many different committees of three students can be formed from a group of four students? 


Solution: To answer this question, we need only find the number of subsets with three ele- 
ments from the set containing the four students. We see that there are four such subsets, one 
for each of the four students, because choosing three students is the same as choosing one of 
the four students to leave out of the group. This means that there are four ways to choose the 
three students for the committee, where the order in which these students are chosen does not 
matter. < 


Example 8 illustrates that many counting problems can be solved by finding the number of 
subsets of a particular size of a set with n elements, where n is a positive integer. 

An r-combination of elements of a set is an unordered selection of r elements from the set. 
Thus, an r-combination is simply a subset of the set with r elements. 


Let S be the set {1, 2, 3, 4}. Then {1, 3, 4} is a 3-combination from S. (Note that {4, 1, 3} is the 
same 3-combination as {1, 3, 4}, because the order in which the elements of a set are listed does 
not matter.) < 


The number of r-combinations of a set with n distinct elements is denoted by C(n, r). Note 
that C(n, r) is also denoted by (”) and is called a binomial coefficient. We will learn where this 
terminology comes from in Section 6.4. 


We see that C(4, 2) = 6, because the 2-combinations of {a, b, c, d} are the six subsets {a, b}, 
{a,c}, {a, d}, {b,c}, {b, d}, and {c,d}. < 


We can determine the number of r-combinations of a set with n elements using the formula 
for the number of r-permutations of a set. To do this, note that the r-permutations of a set can be 
obtained by first forming r-combinations and then ordering the elements in these combinations. 
The proof of Theorem 2, which gives the value of C(n, r), is based on this observation. 


The number of r-combinations of a set with n elements, where n is a nonnegative integer and 
r is an integer with O < r < n, equals 


n! 
C(n, r) = ran! 
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EXAMPLE 11 


Proof: The P(n,r) r-permutations of the set can be obtained by forming the C(n, r) 
r-combinations of the set, and then ordering the elements in each r-combination, which can 
be done in P(r, r) ways. Consequently, by the product rule, 


P(n, r) = C(v,r)- P(r, r). 
This implies that 


P(n, r) _ n/n- _ n! 
Pan r/C) rin- 


C(n, r) = 


We can also use the division rule for counting to construct a proof of this theorem. Because the 
order of elements in a combination does not matter and there are P(r, r) ways to order r elements 
in an r-combination of n elements, each of the C(n, r) r-combinations of a set with n elements 


corresponds to exactly P(r, r) r-permutations. Hence, by the division rule, C(n, r) = aan which 
RT. 


implies as before that C(n, r) = L < 
r!(n—r)! 

The formula in Theorem 2, although explicit, is not helpful when C(n, r) is computed for 
large values of n and r. The reasons are that it is practical to compute exact values of factorials 
exactly only for small integer values, and when floating point arithmetic is used, the formula in 
Theorem 2 may produce a value that is not an integer. When computing C(n, r), first note that 
when we cancel out (n — r)! from the numerator and denominator of the expression for C(n, r) 
in Theorem 2, we obtain 


n! n(n— 1) (n-=r+ 1) 
cs ae er a 


Consequently, to compute C(n, r) you can cancel out all the terms in the larger factorial in the 
denominator from the numerator and denominator, then multiply all the terms that do not cancel 
in the numerator and finally divide by the smaller factorial in the denominator. [When doing this 
calculation by hand, instead of by machine, it is also worthwhile to factor out common factors in 
the numerator n(n — 1) --- (n — r + 1) and in the denominator r!.] Note that many computational 
programs can be used to find C(n, r). [Such functions may be called choose(n, k) or binom(n, k).] 

Example 11 illustrates how C(n, k) is computed when k is relatively small compared to n 
and when k is close to n. It also illustrates a key identity enjoyed by the numbers C(n, k). 


How many poker hands of five cards can be dealt from a standard deck of 52 cards? Also, how 
many ways are there to select 47 cards from a standard deck of 52 cards? 


Solution: Because the order in which the five cards are dealt from a deck of 52 cards does not 
matter, there are 


52! 


different hands of five cards that can be dealt. To compute the value of C(52, 5), first divide the 
numerator and denominator by 47! to obtain 


52 -51-50-49-48 


Casa el 


This expression can be simplified by first dividing the factor 5 in the denominator into the 
factor 50 in the numerator to obtain a factor 10 in the numerator, then dividing the factor 4 in 
the denominator into the factor 48 in the numerator to obtain a factor of 12 in the numerator, 


COROLLARY 2 


Definition 1 


Combinatorial proofs 
are almost always much 
shorter and provide 
more insights than 
proofs based on 
algebraic manipulation. 
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then dividing the factor 3 in the denominator into the factor 51 in the numerator to obtain a factor 
of 17 in the numerator, and finally, dividing the factor 2 in the denominator into the factor 52 in 
the numerator to obtain a factor of 26 in the numerator. We find that 


C(52, 5) = 26-17- 10-49-12 = 2,598,960. 


Consequently, there are 2,598,960 different poker hands of five cards that can be dealt from a 
standard deck of 52 cards. 
Note that there are 


52! 


2, 47) = 2 
C02, 4) = Frisi 


different ways to select 47 cards from a standard deck of 52 cards. We do not need to compute 
this value because C(52, 47) = C(52, 5). (Only the order of the factors 5! and 47! is different in 
the denominators in the formulae for these quantities.) It follows that there are also 2,598,960 
different ways to select 47 cards from a standard deck of 52 cards. < 


In Example 11 we observed that C(52, 5) = C(52, 47). This is not surprising because se- 
lecting five cards out of 52 is the same as selecting the 47 that we leave out. The identity 
C(52, 5) = C(52, 47) is a special case of the useful identity for the number of 7-combinations of 
a set given in Corollary 2. 


Let n and r be nonnegative integers with r < n. Then C(n, r) = C(n, n — r). 


Proof: From Theorem 2 it follows that 


_ n! 
OPUS es 
and 
n! n! 
Ue- t= aaa Gn 
Hence, C(n, r) = C(n, n — r). < 


We can also prove Corollary 2 without relying on algebraic manipulation. Instead, we can 
use a combinatorial proof. We describe this important type of proof in Definition 1. 


A combinatorial proof of an identity is a proof that uses counting arguments to prove that 
both sides of the identity count the same objects but in different ways or a proof that is based 
on showing that there is a bijection between the sets of objects counted by the two sides of 
the identity. These two types of proofs are called double counting proofs and bijective proofs, 
respectively. 


Many identities involving binomial coefficients can be proved using combinatorial proofs. We 
now show how to prove Corollary 2 using a combinatorial proof. We will provide both a double 
counting proof and a bijective proof, both based on the same basic idea. 
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EXAMPLE 13 


EXAMPLE 14 


EXAMPLE 15 


Proof: We will use a bijective proof to show that C(n, r) = C(n, n — r) for all integers n and 
r with O <r <n. Suppose that S is a set with n elements. The function that maps a subset A 
of S to A is a bijection between subsets of S with r elements and subsets with n — r elements 
(as the reader should verify). The identity C(n, r) = C(n, n — r) follows because when there is a 
bijection between two finite sets, the two sets must have the same number of elements. 
Alternatively, we can reformulate this argument as a double counting proof. By definition, 
the number of subsets of S with r elements equals C(n, r). But each subset A of S is also deter- 
mined by specifying which elements are not in A, and so are in A. Because the complement of 
a subset of S with r elements has n — r elements, there are also C(n, n — r) subsets of S with r 
elements. It follows that C(n, r) = C(n,n — r). < 


How many ways are there to select five players from a 10-member tennis team to make a trip to 
a match at another school? 


Solution: The answer is given by the number of 5-combinations of a set with 10 elements. By 
Theorem 2, the number of such combinations is 


10! 
C(10, 5) = S51 252. 
< 


A group of 30 people have been trained as astronauts to go on the first mission to Mars. How 
many ways are there to select a crew of six people to go on this mission (assuming that all crew 
members have the same job)? 


Solution: The number of ways to select a crew of six from the pool of 30 people is the number 
of 6-combinations of a set with 30 elements, because the order in which these people are chosen 
does not matter. By Theorem 2, the number of such combinations is 


30! 30-29-28 - 27-26-25 


C(30, 6) = —— = 
( ) 6! 24! 6-5-4-3-2-1 


= 593,775; 


How many bit strings of length n contain exactly r 1s? 


Solution: The positions of r 1s in a bit string of length n form an r-combination of the set 
{1, 2,3, ..., n}. Hence, there are C(n, r) bit strings of length n that contain exactly r Is. < 


Suppose that there are 9 faculty members in the mathematics department and 11 in the com- 
puter science department. How many ways are there to select a committee to develop a discrete 
mathematics course at a school if the committee is to consist of three faculty members from the 
mathematics department and four from the computer science department? 


Solution: By the product rule, the answer is the product of the number of 3-combinations 
of a set with nine elements and the number of 4-combinations of a set with 11 elements. By 
Theorem 2, the number of ways to select the committee is 


9! 11! 
C9, 3). C1, 4) = 3161 pmi 84 - 330 = 27,720. 


Exercises 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


. List all the permutations of {a, b, c}. 
. How many different permutations are there of the set 


{a,b,c,d,e,f, g}? 


. How many permutations of {a, b, c, d, e, f, g } end with a? 
. Let $= {1, 2, 3,4, 5}. 


a) List all the 3-permutations of S. 
b) List all the 3-combinations of S. 


. Find the value of each of these quantities. 


a) P(6, 3) b) P(6, 5) 
c) P(8, 1) d) P(8, 5) 
e) P(8, 8) f) P10, 9) 
. Find the value of each of these quantities. 
a) C6, 1) b) C6, 3) 
c) C(8, 4) d) C(8, 8) 
e) C(8, 0) f) C(12, 6) 


. Find the number of 5-permutations of a set with nine el- 


ements. 


. In how many different orders can five runners finish a 


race if no ties are allowed? 


. How many possibilities are there for the win, place, and 


show (first, second, and third) positions in a horse race 
with 12 horses if all orders of finish are possible? 

There are six different candidates for governor of a state. 
In how many different orders can the names of the can- 
didates be printed on a ballot? 

How many bit strings of length 10 contain 

a) exactly four 1s? 

b) at most four 1s? 

c) at least four 1s? 

d) an equal number of Os and 1s? 

How many bit strings of length 12 contain 

a) exactly three 1s? 

b) at most three 1s? 

c) at least three 1s? 

d) an equal number of Os and 1s? 

A group contains n men and n women. How many ways 
are there to arrange these people in a row if the men and 
women alternate? 

In how many ways can a set of two positive integers less 
than 100 be chosen? 

In how many ways can a set of five letters be selected 
from the English alphabet? 

How many subsets with an odd number of elements does 
a set with 10 elements have? 

How many subsets with more than two elements does a 
set with 100 elements have? 

A coin is flipped eight times where each flip comes up 
either heads or tails. How many possible outcomes 

a) are there in total? 

b) contain exactly three heads? 

c) contain at least three heads? 

d) contain the same number of heads and tails? 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


27. 
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A coin is flipped 10 times where each flip comes up either 

heads or tails. How many possible outcomes 

a) are there in total? 

b) contain exactly two heads? 

c) contain at most three tails? 

d) contain the same number of heads and tails? 

How many bit strings of length 10 have 

a) exactly three 0s? 

b) more Os than 1s? 

c) at least seven 1s? 

d) at least three 1s? 

How many permutations of the letters ABCDEFG contain 

a) the string BCD? 

b) the string CFGA? 

c) the strings BA and GF? 

d) the strings ABC and DE? 

e) the strings ABC and CDE? 

f) the strings CBA and BED? 

How many permutations of the letters ABCDEFGH con- 

tain 

a) the string ED? 

b) the string CDE? 

c) the strings BA and FGH? 

d) the strings AB, DE, and GH? 

e) the strings CAB and BED? 

f) the strings BCA and ABF? 

How many ways are there for eight men and five women 

to stand in a line so that no two women stand next to each 

other? [Hint: First position the men and then consider 

possible positions for the women. ] 

How many ways are there for 10 women and six men to 

stand in a line so that no two men stand next to each other? 

[Hint: First position the women and then consider possi- 

ble positions for the men.] 

How many ways are there for four men and five women 

to stand in a line so that 

a) all men stand together? 

b) all women stand together? 

How many ways are there for three penguins and six 

puffins to stand in a line so that 

a) all puffins stand together? 

b) all penguins stand together? 

One hundred tickets, numbered 1, 2, 3, ..., 100, are sold 

to 100 different people for a drawing. Four different 

prizes are awarded, including a grand prize (a trip to 

Tahiti). How many ways are there to award the prizes if 

a) there are no restrictions? 

b) the person holding ticket 47 wins the grand prize? 

c) the person holding ticket 47 wins one of the prizes? 

d) the person holding ticket 47 does not win a prize? 

e) the people holding tickets 19 and 47 both win prizes? 

f) the people holding tickets 19, 47, and 73 all win 
prizes? 
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28. 


29. 


30. 


*31. 


32. 


33. 


34. 


g) the people holding tickets 19, 47, 73, and 97 all win 
prizes? 

h) none of the people holding tickets 19, 47, 73, and 97 
wins a prize? 

i) the grand prize winner is a person holding ticket 19, 
47, 73, or 97? 

j) the people holding tickets 19 and 47 win prizes, but 
the people holding tickets 73 and 97 do not win 
prizes? 

Thirteen people on a softball team show up for a game. 


a) How many ways are there to choose 10 players to take 
the field? 

b) How many ways are there to assign the 10 positions 
by selecting players from the 13 people who show up? 

c) Ofthe 13 people who show up, three are women. How 
many ways are there to choose 10 players to take the 
field if at least one of these players must be a woman? 


A club has 25 members. 


a) How many ways are there to choose four members of 
the club to serve on an executive committee? 

b) How many ways are there to choose a president, vice 
president, secretary, and treasurer of the club, where 
no person can hold more than one office? 


A professor writes 40 discrete mathematics true/false 
questions. Of the statements in these questions, 17 are 
true. If the questions can be positioned in any order, how 
many different answer keys are possible? 

How many 4-permutations of the positive integers not ex- 
ceeding 100 contain three consecutive integers k, k + 1, 
k + 2, in the correct order 


a) where these consecutive integers can perhaps be sep- 
arated by other integers in the permutation? 

b) where they are in consecutive positions in the permu- 
tation? 

Seven women and nine men are on the faculty in the 

mathematics department at a school. 


a) How many ways are there to select a committee of 
five members of the department if at least one woman 
must be on the committee? 

b) How many ways are there to select a committee of five 
members of the department if at least one woman and 
at least one man must be on the committee? 


The English alphabet contains 21 consonants and five 

vowels. How many strings of six lowercase letters of the 

English alphabet contain 

a) exactly one vowel? 

b) exactly two vowels? 

c) at least one vowel? 

d) at least two vowels? 

How many strings of six lowercase letters from the En- 

glish alphabet contain 

a) the letter a? 

b) the letters a and b? 

c) the letters a and b in consecutive positions with a pre- 
ceding b, with all the letters distinct? 

d) the letters a and b, where a is somewhere to the left 
of b in the string, with all the letters distinct? 


35. 


36. 


37. 


38. 


39. 


40. 


41. 


Suppose that a department contains 10 men and 15 
women. How many ways are there to form a committee 
with six members if it must have the same number of men 
and women? 


Suppose that a department contains 10 men and 15 
women. How many ways are there to form a committee 
with six members if it must have more women than men? 


How many bit strings contain exactly eight Os and 10 1s 
if every 0 must be immediately followed by a 1? 


How many bit strings contain exactly five Os and 14 1s if 
every 0 must be immediately followed by two 1s? 


How many bit strings of length 10 contain at least three 
1s and at least three Os? 


How many ways are there to select 12 countries in the 
United Nations to serve on a council if 3 are selected from 
a block of 45, 4 are selected from a block of 57, and the 
others are selected from the remaining 69 countries? 


How many license plates consisting of three letters fol- 
lowed by three digits contain no letter or digit twice? 


A circular r-permutation of n people is a seating of r of 
these n people around a circular table, where seatings are con- 
sidered to be the same if they can be obtained from each other 
by rotating the table. 


42. 
43. 


44. 


45. 


*46. 


*47. 


*48. 


Find the number of circular 3-permutations of 5 people. 


Find a formula for the number of circular r-permutations 
of n people. 


Find a formula for the number of ways to seat r of n peo- 
ple around a circular table, where seatings are consid- 
ered the same if every person has the same two neighbors 
without regard to which side these neighbors are sitting 
on. 


How many ways are there for a horse race with three 
horses to finish if ties are possible? [Note: Two or three 
horses may tie.] 


How many ways are there for a horse race with four 
horses to finish if ties are possible? [Note: Any number 
of the four horses may tie.] 


There are six runners in the 100-yard dash. How many 
ways are there for three medals to be awarded if ties 
are possible? (The runner or runners who finish with the 
fastest time receive gold medals, the runner or runners 
who finish with exactly one runner ahead receive silver 
medals, and the runner or runners who finish with exactly 
two runners ahead receive bronze medals.) 


This procedure is used to break ties in games in the cham- 
pionship round of the World Cup soccer tournament. 
Each team selects five players in a prescribed order. Each 
of these players takes a penalty kick, with a player from 
the first team followed by a player from the second team 
and so on, following the order of players specified. If the 
score is still tied at the end of the 10 penalty kicks, this 
procedure is repeated. If the score is still tied after 20 
penalty kicks, a sudden-death shootout occurs, with the 
first team scoring an unanswered goal victorious. 
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a) How many different scoring scenarios are possible if the game is settled in the second round of 10 penalty 
the game is settled in the first round of 10 penalty kicks? 
kicks, where the round ends once it is impossible for a 


team to equal the number of goals scored by the other c) How many scoring scenarios are possible for the full 
team? set of penalty kicks if the game is settled with no more 

b) How ‘many different scoring scenarios for the first than 10 total additional kicks after the two rounds of 
and second groups of penalty kicks are possible if five kicks for each team? 


Binomial Coefficients and Identities 


Links » 


EXAMPLE 1 


THEOREM 1 


As we remarked in Section 6.3, the number of r-combinations from a set with n elements is 
often denoted by (ie This number is also called a binomial coefficient because these numbers 
occur as coefficients in the expansion of powers of binomial expressions such as (a + b)”. We 
will discuss the binomial theorem, which gives a power of a binomial expression as a sum of 
terms involving binomial coefficients. We will prove this theorem using a combinatorial proof. 
We will also show how combinatorial proofs can be used to establish some of the many different 
identities that express relationships among binomial coefficients. 


6.4.1 The Binomial Theorem 


The binomial theorem gives the coefficients of the expansion of powers of binomial expressions. 
A binomial expression is simply the sum of two terms, such as x + y. (The terms can be products 
of constants and variables, but that does not concern us here.) 

Example 1 illustrates how the coefficients in a typical expansion can be found and prepares 
us for the statement of the binomial theorem. 


The expansion of (x + y)? can be found using combinatorial reasoning instead of multiplying the 
three terms out. When (x + y)? = (x +y)(x + y)(x + y) is expanded, all products of a term in the 
first sum, a term in the second sum, and a term in the third sum are added. Terms of the form x°, 
xy, xy*, and y? arise. To obtain a term of the form x°, an x must be chosen in each of the sums, and 
this can be done in only one way. Thus, the x° term in the product has a coefficient of 1. To obtain a 
term of the form xy, an x must be chosen in two of the three sums (and consequently a y in the other 
sum). Hence, the number of such terms is the number of 2-combinations of three objects, namely, 
Cie Similarly, the number of terms of the form xy? is the number of ways to pick one of the three 
sums to obtain an x (and consequently take a y from each of the other two sums). This can be done 
in £) ways. Finally, the only way to obtain a y? term is to choose the y for each of the three sums 


in the product, and this can be done in exactly one way. Consequently, it follows that 


x+y = AHVE E +y) = ax + xy + yx + yy) +y) 
= XXX + XXY + xyx + xyy + yxx + yxy + yyx + yyy 
= + 3x*y + 3y +y. 


We now state the binomial theorem. 


THE BINOMIAL THEOREM  Letx andy be variables, and let n be a nonnegative integer. 
Then 


n . n n-j,j n n n n— n n— n n 
PG eal) ae eee 
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EXAMPLE 3 


EXAMPLE 4 


Proof: We use a combinatorial proof. The terms in the product when it is expanded are of the 
form x"/y for j = 0, 1, 2, ..., n. To count the number of terms of the form x"’y/, note that to 
obtain such a term it is necessary to choose n — j xs from the n binomial factors (so that the 
other j terms in the product are ys). Therefore, the coefficient of x”~y/ is (i) which is equal 
to (C). This proves the theorem. < 
Some computational uses of the binomial theorem are illustrated in Examples 2—4. 


What is the expansion of (x + yy? 


Solution: From the binomial theorem it follows that 


4 


(x +y = 2 Pi 


j=0 


OOOO OE 


= xf + 4y + Oxy + Axy? yi. 


< 
What is the coefficient of x!?y!? in the expansion of (x + y)*>? 
Solution: From the binomial theorem it follows that this coefficient is 
! 
oa = a = 5,200,300. 
13 13! 12! ä 


What is the coefficient of x!?y! in the expansion of (2x — 3y)?>? 


Solution: First, note that this expression equals (2x + (—3y))*>. By the binomial theorem, we 
have 


Qx+ (3 =¥ © Janay, 
j=0 


Consequently, the coefficient of x!y!? 


in the expansion is obtained when j = 13, namely, 


(Beea! =- 25! 912313 


Note that another way to find the solution is to first use the binomial theorem to see that 
25 
(u +v) = by (7°) 2 
joo NJ 
Setting u = 2x and v = —3y in this equation yields the same result. < 


We can prove some useful identities using the binomial theorem, as Corollaries 1, 2, and 3 
demonstrate. 


COROLLARY 1 


COROLLARY 2 


6.4 Binomial Coefficients and Identities 


Let n be a nonnegative integer. Then 


H 


Proof: Using the binomial theorem with x = 1 and y = 1, we see that 


x#=(1+1= 5 (7) peyn-* = 5 @ 


k=0 k=0 


This is the desired result. 


There is also a nice combinatorial proof of Corollary 1, which we now present. 


439 


Proof: A set with n elements has a total of 2” different subsets. Each subset has zero elements, 


one element, two elements, ..., or n elements in it. There are ©) subsets with zero elements, (5) 

subsets with one element, (3) subsets with two elements, ..., and (") subsets with n elements. 
n 

Therefore, 


ye 


counts the total number of subsets of a set with n elements. By equating the two formulas we 


have for the number of subsets of a set with n elements, we see that 


Let n be a positive integer. Then 


coz) = 0. 
2o 


Proof: When we use the binomial theorem with x = —1 and y = 1, we see that 
0=0=(-D)+1}= Yep’ = V(t 
(C1) +1) Len 24g) 


This proves the corollary. 


Remark: Corollary 2 implies that 
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Let n be a nonnegative integer. Then 
I) a8, 


Proof: We recognize that the left-hand side of this formula is the expansion of (1 + 2)” provided 
by the binomial theorem. Therefore, by the binomial theorem, we see that 


(1+2)"= 5 (a = 5 ag 


k=0 k=0 
Hence 
r (") aS, 


6.4.2 Pascal’s Identity and Triangle 


The binomial coefficients satisfy many different identities. We introduce one of the most im- 
portant of these now. 


PASCAL’S IDENTITY Letn and k be positive integers with n > k. Then 
ol \ n n 
Cae 


Proof: We will use a combinatorial proof. Suppose that T is a set containing n + 1 elements. Let 
a be an element in 7, and let S = T — {a}. Note that there are ("+ subsets of T containing k 
elements. However, a subset of T with k elements either contains a together with k — 1 elements 


of S, or contains k elements of S and does not contain a. Because there are ( a subsets of 
n 


k — 1 elements of S, there are ( i ) subsets of k elements of T that contain a. And there are (X) 


subsets of k elements of T that do not contain a, because there are (") subsets of k elements of 
S. Consequently, 


Ced) ; 


Remark: It is also possible to prove this identity by algebraic manipulation from the formula 
for (”) (see Exercise 23). 


Remark: Pascal’s identity, together with the initial conditions (o) = (”) = | for all integers n, 
can be used to recursively define binomial coefficients. This recursive definition is useful in the 
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FIGURE 1 Pascalľ’s triangle. 


Source: National Library of 
Medicine 


computation of binomial coefficients because only addition, and not multiplication, of integers 
is needed to use this recursive definition. 


Pascal’s identity is the basis for a geometric arrangement of the binomial coefficients in a 
triangle, as shown in Figure 1. 
The nth row in the triangle consists of the binomial coefficients 


(7) k=0,1,....7 


This triangle is known as Pascal’s triangle, named after the French mathematician Blaise 
Pascal. Pascal’s identity shows that when two adjacent binomial coefficients in this triangle 
are added, the binomial coefficient in the next row between these two coefficients is produced. 

Pascal’s triangle has a long and ancient history, predating Pascal by many centuries. In the 
East, binomial coefficients and Pascal’s identity were known in the second century B.C.E. by the 
Indian mathematician Pingala. Later, Indian mathematicians included commentaries relating 
to Pascal’s triangle in their books written in the first half of the last millennium. The Persian 


BLAISE PASCAL (1623-1662) Blaise Pascal was taught by his father, a tax collector in Rouen, France. He 
exhibited his talents at an early age, although his father, who had made discoveries in analytic geometry, kept 
mathematics books away from him to encourage other interests. At 16 Pascal discovered an important result 
concerning conic sections. At 18 he designed a calculating machine, which he built and sold. Pascal, along 
with Fermat, laid the foundations for the modern theory of probability. In this work, he made new discoveries 
concerning what is now called Pascal’s triangle. In 1654, Pascal abandoned his mathematical pursuits to de- 
vote himself to theology. After this, he returned to mathematics only once. One night, distracted by a severe 
toothache, he sought comfort by studying the mathematical properties of the cycloid. Miraculously, his pain 
subsided, which he took as a sign of divine approval of the study of mathematics. 
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mathematician Al-Karaji and the multitalented Omar Khayyám wrote about Pascal’s triangle in 
the eleventh and twelfth centuries, respectively; in Iran, Pascal’s triangle is known as Khayyam’s 
triangle. The triangle was known by the Chinese mathematician Jia Xian in the eleventh century 
and was written about in the 13th century by Yang Hui; in Chinese Pascal’s triangle is often 
known as Yang Hui’s triangle. 

In the West, Pascal’s triangle appears on the frontispiece of a 1527 book on business calcula- 
tion written by the German scholar Petrus Apianus. In Italy, Pascal’s triangle is called Tartaglia’s 
triangle, after the Italian mathematician Niccolò Fontana Tartaglia who published the first few 
rows of the triangle in 1556. In his book Traité du triangle arithmétique, published posthu- 
mously 1665, Pascal presented results about Pascal’s triangle and used them to solve probability 
theory problems. Later French mathematicians named this triangle after Pascal; in 1730 Abra- 
ham de Moivre coined the name “Pascal’s Arithmetic Triangle,’ which later became “Pascal’s 
Triangle.” 


6.4.3 Other Identities Involving Binomial Coefficients 


We conclude this section with combinatorial proofs of two of the many identities enjoyed by 
the binomial coefficients. 


VANDERMONDE’S IDENTITY Let m, n, and r be nonnegative integers with r not ex- 
ceeding either m or n. Then 


7") 2 (22) 


Remark: This identity was discovered by mathematician Alexandre-Théophile Vandermonde 
in the eighteenth century. 


Proof: Suppose that there are m items in one set and n items in a second set. Then the total 
number of ways to pick r elements from the union of these sets is (” a 

Another way to pick r elements from the union is to pick k elements from the second set 
and then r — k elements from the first set, where k is an integer with 0 < k < r. Because there 
are (X) ways to choose k elements from the second set and (” J ways to choose r — k elements 


from the first set, the product rule tells us that this can be done in (”") (7 
total number of ways to pick r elements from the union also equals };; _, (") (7). 

We have found two expressions for the number of ways to pick r elements from the 
union of a set with m items and a set with n items. Equating them gives us Vandermonde’s 


identity. < 


) ways. Hence, the 


Corollary 4 follows from Vandermonde’s identity. 


ALEXANDRE-THÉOPHILE VANDERMONDE (1735-1796) Because Alexandre-Théophile Vandermonde 
was a sickly child, his physician father directed him to a career in music. However, he later developed an interest 
in mathematics. His complete mathematical work consists of four papers published in 1771-1772. These papers 
include fundamental contributions on the roots of equations, on the theory of determinants, and on the knight’s 
tour problem (introduced in the exercises in Section 10.5). Vandermonde’s interest in mathematics lasted for 
only 2 years. Afterward, he published papers on harmony, experiments with cold, and the manufacture of steel. 
He also became interested in politics, joining the cause of the French revolution and holding several different 
positions in government. 


COROLLARY 4 


THEOREM 4 


Exercises 


1. Find the expansion of (x + y)* 5. How many terms are there in the expansion of (x + y 
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If n is a nonnegative integer, then 
n 2 
( 4 E \k 
Proof: We use Vandermonde’s identity with m = r = n to obtain 


n n 2 
2n n n n 
Or) BO M)-Z) 
The last equality was obtained using the identity (") = (e AE < 
We can prove combinatorial identities by counting bit strings with different properties, as 
the proof of Theorem 4 will demonstrate. 


Let n and r be nonnegative integers with r < n. Then 
n+1)\ _ - J 
GDR 


Proof: We use a combinatorial proof. By Example 14 in Section 6.3, the left-hand side, ( 
counts the bit strings of length n + 1 containing r + 1 ones. 

We show that the right-hand side counts the same objects by considering the cases corre- 
sponding to the possible locations of the final 1 in a string with r + 1 ones. This final one must 
occur at position r + 1, r + 2, ..., or n + 1. Furthermore, if the last one is the kth bit there must 
be r ones among the first k — 1 positions. Consequently, by Example 14 in Section 6.3, there 
are "y ') such bit strings. Summing over k with r+ 1 < k < n + 1, we find that there are 


n+1 n 
x) 20) 
x ( p j=r r 
bit strings of length n containing exactly r + 1 ones. (Note that the last step follows from the 


change of variables j = k — 1.) Because the left-hand side and the right-hand side count the 
same objects, they are equal. This completes the proof. < 


n+l) 
r+1/’ 


100 


a) using combinatorial reasoning, as in Example 1. after like terms are collected? 
b) using the binomial theorem. 6. What is the coefficient of x” in (1 + x)!!? 
2. Find the expansion of (x + y)° 7. What is the coefficient of x? in (2 — x)!°? 
a) using combinatorial reasoning, as in Example 1. 8. What is the coefficient of x®y? in the expansion of 
b) using the binomial theorem. (3x + 2y)!7? 
3. Find the expansion of (x + y)°. 9. What is the coefficient of x!°!y? in the expansion of 
4. Find the coefficient of xy* in (x + y)’. (2x — 3y)? 
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10. 


11. 


12. 


13. 


*14. 


*T5. 


16. 


17. 


18. 


19. 


20. 


S21 


22. 


23. 
24. 


Use the binomial theorem to expand (3x — y*)* into a sum 
of terms of the form cx4y? , where c is a real number and 
a and b are nonnegative integers. 

Use the binomial theorem to expand (3x* — 2y*)° into a 
um of terms of the form cxty?, where c is a real number 
and a and b are nonnegative integers. 


Use the binomial theorem to find the coefficient of x“y’ 
in the expansion of (5x? + 2y*)°, where 


a) a=6,b=9. 
b) a=2,b= 15. 
c) a=3,b=12. 
d) a=12,b=0. 
e) a=8,b=9. 


Use the binomial theorem to find the coefficient of x“y’ 
in the expansion of (2x? — 4y?)’, where 


a) a=9,b=8. 
b) a=8,b=9. 
c) a=0,b= 14. 
d) a=12,b=6. 
e) a=18,b=2. 


Give a formula for the coefficient of x* in the expansion 
of (x + 1/x)!, where k is an integer. 

Give a formula for the coefficient of x* in the expansion 
of (x? — 1/x)!, where k is an integer. 

The row of Pascal’s triangle containing the binomial co- 
efficients lea? 0<k < 10, is: 


1 10 45 120 210 252 210 120 45 10 1 


Use Pascal’s identity to produce the row immediately 

following this row in Pascal’s triangle. 

What is the row of Pascal’s triangle containing the bino- 

mial coefficients (), 0<k<9? 

Show that if n is a positive integer, then 1 = (o) < (") < 

=< (ar = (ira) REE (S) > (") =. 

Show that (") <2” for all positive integers n and all 

integers k with O < k <n. 

a) Use Exercise 18 and Corollary 1 to show that if n is 
an integer greater than 1, then bap p > 2"/n. 

b) Conclude from part (a) that if n is a positive integer, 
then (*") > 4"/2n. 


. Show that if n and k are integers with 1 < k <n, then 


(") < nk J21, 

Suppose that b is an integer with b > 7. Use the bino- 
mial theorem and the appropriate row of Pascal’s triangle 
to find the base-b expansion of (1 D [that is, the fourth 
power of the number (11), in base-b notation]. 
Prove Pascal’s identity, using the formula for ("). 
Suppose that k and n are integers with 1 < k < n. Prove 
the hexagon identity 


(Ge le (en ican) 


which relates terms in Pascal’s triangle that form a 
hexagon. 


I 25, 


26. 


27. 


28. 


29. 


* 30. 


*31, 


32. 


*33. 


*34, 


35. 


* 36. 


Prove that if n and k are integers with 1 < k <n, then 

k(t) =al) 

a) using a combinatorial proof. [Hint: Show that the two 
sides of the identity count the number of ways to se- 
lect a subset with k elements from a set with n ele- 
ments and then an element of this subset. ] 

b) using an algebraic proof based on the formula for (©) 
given in Theorem 2 in Section 6.3. 


Prove the identity (") C) = (7) Cais whenever n, r, and 

k are nonnegative integers with r < n and k <r, 

a) using a combinatorial argument. 

b) using an argument based on the formula for the num- 
ber of r-combinations of a set with n elements. 


Show that if n and k are positive integers, then 


n+1\ _ n 
( i )=o+n(,")) [k 


Use this identity to construct an inductive definition of 
the binomial coefficients. 

Show that if p is a prime and k is an integer such that 
1 <k < p- 1, then p divides (?). 


Let n be a positive integer. Show that 


et ees 


Let n and k be integers with 1 < k < n. Show that 


SOl- Gre- G) 


Prove the hockeystick identity 


ey E eC) 


whenever n and r are positive integers, 


a) using a combinatorial argument. 
b) using Pascal’s identity. 


Show that if n is a positive integer, then C) = 2(5) +n? 


a) using a combinatorial argument. 
b) by algebraic manipulation. 


r $ : n n\ _ =] 

Give a combinatorial proof that J}; k(?) =n, 
[Hint: Count in two ways the number of ways to select 
a committee and to then select a leader of the commit- 


tee.] 


: ‘ : 2 = 
Give a combinatorial proof that )77_, k(?) = n(""). 
[Hint: Count in two ways the number of ways to select a 
committee, with n members from a group of n mathemat- 
ics professors and n computer science professors, such 
that the chairperson of the committee is a mathematics 
professor. ] 

Show that a nonempty set has the same number of sub- 
sets with an odd number of elements as it does subsets 
with an even number of elements. 

Prove the binomial theorem using mathematical induc- 
tion. 


37. 


(0, 0) 


In this exercise we will count the number of paths in the 
xy plane between the origin (0, 0) and point (m, n), where 
m and n are nonnegative integers, such that each path is 
made up of a series of steps, where each step is a move 
one unit to the right or a move one unit upward. (No 
moves to the left or downward are allowed.) Two such 
paths from (0, 0) to (5, 3) are illustrated here. 


(5, 3) 


(5, 3) 


(0, 0) 


38. 


a) Show that each path of the type described can be rep- 
resented by a bit string consisting of m Os and n 1s, 
where a 0 represents a move one unit to the right and 
a | represents a move one unit upward, 

b) Conclude from part (a) that there are rg paths of 
the desired type. 

Use Exercise 37 to give an alternative proof of Corol- 

lary 2 in Section 6.3, which states that (7) = (0 when- 

ever k is an integer with O < k < n. [Hint: Consider the 
number of paths of the type described in Exercise 37 from 

(0, 0) to (n — k, k) and from (0, 0) to (k, n — k).] 


39. 


40. 


41. 


42. 


* 43, 
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Use Exercise 37 to prove Theorem 4. [Hint: Count the 
number of paths with n steps of the type described in Ex- 
ercise 37. Every such path must end at one of the points 
(n—k,k) fork = 0, 1, 2, ..., n.] 


Use Exercise 37 to prove Pascal’s identity. [Hint: Show 
that a path of the type described in Exercise 37 
from (0,0) to (7+1-—k,k) passes through either 
(n+1—k,k—1) or (n -— k, k), but not through both.] 


Use Exercise 37 to prove the hockeystick identity from 
Exercise 31. [Hint: First, note that the number of 
paths from (0, 0) to (n + 1, r) equals anal Second, 
count the number of paths by summing the number of 
these paths that start by going k units upward for k = 
ON 2 ieee al 


Give a combinatorial proof that if n is a positive inte- 
ger then eo |) = n(n + 1)2"~*. [Hint: Show that 
both sides count the ways to select a subset of a set of 
n elements together with two not necessarily distinct el- 
ements from this subset. Furthermore, express the right- 
hand side as n(n — 1)2”-? + n2™!.] 

Determine a formula involving binomial coefficients for 
the nth term of a sequence if its initial terms are those 
listed. [Hint: Looking at Pascal’s triangle will be helpful. 
Although infinitely many sequences start with a speci- 
fied set of terms, each of the following lists is the start of 
a sequence of the type desired. ] 


a) 1,3, 6, 10, 15, 21, 28, 36, 45, 55, 66, ... 

b) 1, 4, 10, 20, 35, 56, 84, 120, 165, 220, ... 

c) 1, 2, 6, 20, 70, 252, 924, 3432, 12870, 48620, ... 

d) 1, 1, 2, 3, 6, 10, 20, 35, 70, 126, ... 

e): My MWS 5,15, 355 159%, aa 

f) 1, 3, 15, 84, 495, 3003, 18564, 116280, 735471, 
4686825, ... 


Generalized Permutations and Combinations 


6.5.1 Introduction 


Links » 


In many counting problems, elements may be used repeatedly. For instance, a letter or digit may 
be used more than once on a license plate. When a dozen donuts are selected, each variety can 
be chosen repeatedly. This contrasts with the counting problems discussed earlier in the chapter 
where we considered only permutations and combinations in which each item could be used at 
most once. In this section we will show how to solve counting problems where elements may 
be used more than once. 

Also, some counting problems involve indistinguishable elements. For instance, to count the 
number of ways the letters of the word SUCCESS can be rearranged, the placement of identical 
letters must be considered. This contrasts with the counting problems discussed earlier where all 
elements were considered distinguishable. In this section we will describe how to solve counting 
problems in which some elements are indistinguishable. 

Moreover, in this section we will explain how to solve another important class of count- 
ing problems, problems involving counting the ways distinguishable elements can be placed in 
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EXAMPLE 1 


THEOREM 1 


EXAMPLE 2 


boxes. An example of this type of problem is the number of different ways poker hands can be 
dealt to four players. 

Taken together, the methods described earlier in this chapter and the methods introduced 
in this section form a useful toolbox for solving a wide range of counting problems. When the 
additional methods discussed in Chapter 8 are added to this arsenal, you will be able to solve a 
large percentage of the counting problems that arise in a wide range of areas of study. 


6.5.2 Permutations with Repetition 

Counting permutations when repetition of elements is allowed can easily be done using the 
product rule, as Example 1 shows. 

How many strings of length r can be formed from the uppercase letters of the English alphabet? 
Solution: By the product rule, because there are 26 uppercase English letters, and because each 


letter can be used repeatedly, we see that there are 26” strings of uppercase English letters of 
length r. < 


The number of r-permutations of a set with n elements when repetition is allowed is given 
in Theorem 1. 


The number of r-permutations of a set of n objects with repetition allowed is n”. 


Proof: There are n ways to select an element of the set for each of the r positions in the 
r-permutation when repetition is allowed, because for each choice all n objects are available. 
Hence, by the product rule there are n” r-permutations when repetition is allowed. < 


6.5.3 Combinations with Repetition 


Consider these examples of combinations with repetition of elements allowed. 


How many ways are there to select four pieces of fruit from a bowl containing apples, oranges, 
and pears if the order in which the pieces are selected does not matter, only the type of fruit and 
not the individual piece matters, and there are at least four pieces of each type of fruit in the 
bowl? 


Solution: To solve this problem we list all the ways possible to select the fruit. There are 15 
ways: 


4 apples 4 oranges 4 pears 

3 apples, 1 orange 3 apples, 1 pear 3 oranges, 1 apple 
3 oranges, 1 pear 3 pears, | apple 3 pears, 1 orange 
2 apples, 2 oranges 2 apples, 2 pears 2 oranges, 2 pears 


2 apples, 1 orange, 1 pear 2 oranges, | apple, 1 pear 2 pears, | apple, 1 orange 


The solution is the number of 4-combinations with repetition allowed from a three-element set, 
{apple, orange, pear}. < 


To solve more complex counting problems of this type, we need a general method for count- 
ing the r-combinations of an n-element set. In Example 3 we will illustrate such a method. 


EXAMPLE 3 
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$100 $50 $20 $10 $5 $2 $1 


FIGURE 1 Cash box with seven types of bills. 


How many ways are there to select five bills from a cash box containing $1 bills, $2 bills, 
$5 bills, $10 bills, $20 bills, $50 bills, and $100 bills? Assume that the order in which the bills 
are chosen does not matter, that the bills of each denomination are indistinguishable, and that 
there are at least five bills of each type. 


Solution: Because the order in which the bills are selected does not matter and seven differ- 
ent types of bills can be selected as many as five times, this problem involves counting 5- 
combinations with repetition allowed from a set with seven elements. Listing all possibilities 
would be tedious, because there are a large number of solutions. Instead, we will illustrate the 
use of a technique for counting combinations with repetition allowed. 

Suppose that a cash box has seven compartments, one to hold each type of bill, as illustrated 
in Figure 1. These compartments are separated by six dividers, as shown in the picture. The 
choice of five bills corresponds to placing five markers in the compartments holding different 
types of bills. Figure 2 illustrates this correspondence for three different ways to select five bills, 
where the six dividers are represented by bars and the five bills by stars. 

The number of ways to select five bills corresponds to the number of ways to arrange six 
bars and five stars in a row with a total of 11 positions. Consequently, the number of ways to 
select the five bills is the number of ways to select the positions of the five stars from the 11 


ES 


apaa E 


*||]**ll*l* 


FIGURE 2 Examples of ways to select five bills. 
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EXAMPLE 4 
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positions. This corresponds to the number of unordered selections of 5 objects from a set of 11 
objects, which can be done in C(11, 5) ways. Consequently, there are 


11! 


ways to choose five bills from the cash box with seven types of bills. < 


Theorem 2 generalizes this discussion. 


There are C(n +r -— 1, r) = C(n+r-—1,n-— 1) r-combinations from a set with n elements 
when repetition of elements is allowed. 


Proof: Each r-combination of a set with n elements when repetition is allowed can be repre- 
sented by a list of n — 1 bars and r stars. The n — 1 bars are used to mark off n different cells, 
with the ith cell containing a star for each time the ith element of the set occurs in the combi- 
nation. For instance, a 6-combination of a set with four elements is represented with three bars 
and six stars. Here 


represents the combination containing exactly two of the first element, one of the second ele- 
ment, none of the third element, and three of the fourth element of the set. 

As we have seen, each different list containing n — 1 bars and r stars corresponds to an r- 
combination of the set with n elements, when repetition is allowed. The number of such lists is 
C(n — 1 +r, r), because each list corresponds to a choice of the r positions to place the r stars 
from the n — 1 + r positions that contain r stars and n — 1 bars. The number of such lists is also 
equal to C(n — 1 + r, n — 1), because each list corresponds to a choice of the n — 1 positions to 
place the n — 1 bars. < 


Examples 4—6 show how Theorem 2 is applied. 


Suppose that a cookie shop has four different kinds of cookies. How many different ways can 
six cookies be chosen? Assume that only the type of cookie, and not the individual cookies or 
the order in which they are chosen, matters. 


Solution: The number of ways to choose six cookies is the number of 6-combinations of a set 
with four elements. From Theorem 2 this equals C(4 + 6 — 1, 6) = C(9, 6). Because 


9-8-7 
C9, 6 = C(Q9, 3) = = 84, 
(9, 6) = C(9, 3) = —5 
there are 84 different ways to choose the six cookies. < 


Theorem 2 can also be used to find the number of solutions of certain linear equations where 
the variables are integers subject to constraints. This is illustrated by Example 5. 


How many solutions does the equation 
x +X +x; = 11 


have, where x,, x,, and x; are nonnegative integers? 


EXAMPLE 6 
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Solution: To count the number of solutions, we note that a solution corresponds to a way of 
selecting 11 items from a set with three elements so that x, items of type one, x, items of type 
two, and x, items of type three are chosen. Hence, the number of solutions is equal to the number 
of 11-combinations with repetition allowed from a set with three elements. From Theorem 2 it 
follows that there are 

13-12 

c(3 + 11-1, 11) = C(13, 11) = C(13, 2) = ore lee 78 
solutions. 

The number of solutions of this equation can also be found when the variables are subject 
to constraints. For instance, we can find the number of solutions where the variables are integers 
with x, > 1, x > 2, and x; > 3. A solution to the equation subject to these constraints corre- 
sponds to a selection of 11 items with x, items of type one, x, items of type two, and x3 items 
of type three, where, in addition, there is at least one item of type one, two items of type two, 
and three items of type three. So, a solution corresponds to a choice of one item of type one, 
two of type two, and three of type three, together with a choice of five additional items of any 
type. By Theorem 2 this can be done in 


CG+5~1,5)=C(,5)=C0,2)= =$ =21 
ways. Thus, there are 21 solutions of the equation subject to the given constraints. < 


Example 6 shows how counting the number of combinations with repetition allowed arises 
in determining the value of a variable that is incremented each time a certain type of nested loop 
is traversed. 


What is the value of k after the following pseudocode has been executed? 


k:=0 
for i; := 1 ton 
for i, := 1 toi, 


for i„ := 1 to ip 
k:=k+1 


Solution: Note that the initial value of k is O and that 1 is added to k each time the nested loop 
is traversed with a sequence of integers i, i», ..., i, such that 
= 'm = *m-1 sexsi <n. 

The number of such sequences of integers is the number of ways to choose m integers from 
{1,2,... n}, with repetition allowed. (To see this, note that once such a sequence has been 
selected, if we order the integers in the sequence in nondecreasing order, this uniquely defines 
an assignment of i,,, i,,;, --., i}. Conversely, every such assignment corresponds to a unique 
unordered set.) Hence, from Theorem 2, it follows that k = C(n + m — 1, m) after this code has 
been executed. < 


The formulae for the numbers of ordered and unordered selections of r elements, chosen 
with and without repetition allowed from a set with n elements, are shown in Table 1. 
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TABLE 1 Combinations and Permutations With 
and Without Repetition. 


: n! 
r-permutations No -—— 
(n—r)! 
Boe ead n! 
r-combinations No — 
r!(n—r)! 
r-permutations Yes n” 
PENEN n+r-—1)! 
r-combinations Yes Grr- 
r!(n— 1)! 


6.5.4 Permutations with Indistinguishable Objects 


Some elements may be indistinguishable in counting problems. When this is the case, care must 
be taken to avoid counting things more than once. Consider Example 7. 


How many different strings can be made by reordering the letters of the word SUCCESS? 


Solution: Because some of the letters of SUCCESS are the same, the answer is not given by the 
number of permutations of seven letters. This word contains three Ss, two Cs, one U, and one E. 
To determine the number of different strings that can be made by reordering the letters, first note 
that the three Ss can be placed among the seven positions in C(7, 3) different ways, leaving four 
positions free. Then the two Cs can be placed in C(4, 2) ways, leaving two free positions. The U 
can be placed in C(2, 1) ways, leaving just one position free. Hence E can be placed in C(1, 1) 
way. Consequently, from the product rule, the number of different strings that can be made is 


ToO 4 2 o1 
CO SICER ACG, EUL AY 314! 212! 111! 110! 
7! 
= 32T! 


= 420. < 


We can prove Theorem 3 using the same sort of reasoning as in Example 7. 


The number of different permutations of n objects, where there are n, indistinguishable ob- 
jects of type 1, n, indistinguishable objects of type 2, ..., and n, indistinguishable objects of 
type k, is 


n! 
ni! n! eeng! 


Proof: To determine the number of permutations, first note that the n, objects of type one can 
be placed among the n positions in C(n, n,) ways, leaving n — n; positions free. Then the objects 
of type two can be placed in C(n — nı, n) ways, leaving n — n; — n, positions free. Continue 
placing the objects of type three, ..., type k — 1, until at the last stage, n, objects of type k can 
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be placed in C(n — n; — ny — +++ — Nng- ng) ways. Hence, by the product rule, the total number 
of different permutations is 


C(n, ny )C(n — ny, Ny) = C(n — Ny = + — ng- Ny) 
= n! (n—n,)! . (n=n =*= — ng)! 
~ min—n)! ny! (n—n, -m )! n,!0! 
_ n! 
nlme! < 


6.5.5 Distributing Objects into Boxes 


Many counting problems can be solved by enumerating the ways objects can be placed into 
boxes (where the order these objects are placed into the boxes does not matter). The objects can 
be either distinguishable, that is, different from each other, or indistinguishable, that is, consid- 
ered identical. Distinguishable objects are sometimes said to be labeled, whereas indistinguish- 
able objects are said to be unlabeled. Similarly, boxes can be distinguishable, that is, different, 
or indistinguishable, that is, identical. Distinguishable boxes are often said to be labeled, while 
indistinguishable boxes are said to be unlabeled. When you solve a counting problem using the 
model of distributing objects into boxes, you need to determine whether the objects are dis- 
tinguishable and whether the boxes are distinguishable. Although the context of the counting 
problem makes these two decisions clear, counting problems are sometimes ambiguous and it 
may be unclear which model applies. In such a case it is best to state whatever assumptions you 
are making and explain why the particular model you choose conforms to your assumptions. 

We will see that there are closed formulae for counting the ways to distribute objects, 
distinguishable or indistinguishable, into distinguishable boxes. We are not so lucky when we 
count the ways to distribute objects, distinguishable or indistinguishable, into indistinguishable 
boxes; there are no closed formulae to use in these cases. 


Remark: A closed formula is an expression that can be evaluated using a finite number of 
operations and that includes numbers, variables, and values of functions, where the operations 
and functions belong to a generally accepted set that can depend on the context. In this book, we 
include the usual arithmetic operations, rational powers, exponential and logarithmic functions, 
trigonometric functions, and the factorial function. We do not allow infinite series to be included 
in closed formulae. 


DISTINGUISHABLE OBJECTS AND DISTINGUISHABLE BOXES We first consider the 
case when distinguishable objects are placed into distinguishable boxes. Consider Example 8 
in which the objects are cards and the boxes are hands of players. 


How many ways are there to distribute hands of 5 cards to each of four players from the standard 
deck of 52 cards? 


Solution: We will use the product rule to solve this problem. To begin, note that the first player 
can be dealt 5 cards in C(52, 5) ways. The second player can be dealt 5 cards in C(47, 5) ways, 
because only 47 cards are left. The third player can be dealt 5 cards in C(42, 5) ways. Finally, 
the fourth player can be dealt 5 cards in C(37, 5) ways. Hence, the total number of ways to deal 
four players 5 cards each is 
52! 47! 42! 37! 
C(52, 5)C(47, 5)C(42, 5)C(37, 5) = ——  —— z i —— 
( de Met et ) 47!5! 42!5! 37!5! 32!5! 
_ 52! 
~ 5151515132!" <4 
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Remark: The solution to Example 8 equals the number of permutations of 52 objects, with 
5 indistinguishable objects of each of four different types, and 32 objects of a fifth type. This 
equality can be seen by defining a one-to-one correspondence between permutations of this 
type and distributions of cards to the players. To define this correspondence, first order the 
cards from 1 to 52. Then cards dealt to the first player correspond to the cards in the positions 
assigned to objects of the first type in the permutation. Similarly, cards dealt to the second, 
third, and fourth players, respectively, correspond to cards in the positions assigned to objects 
of the second, third, and fourth type, respectively. The cards not dealt to any player correspond 
to cards in the positions assigned to objects of the fifth type. The reader should verify that this 
is a one-to-one correspondence. 


Example 8 is a typical problem that involves distributing distinguishable objects into dis- 
tinguishable boxes. The distinguishable objects are the 52 cards, and the five distinguishable 
boxes are the hands of the four players and the rest of the deck. Counting problems that involve 
distributing distinguishable objects into boxes can be solved using Theorem 4. 


The number of ways to distribute n distinguishable objects into k distinguishable boxes so 
that n; objects are placed into box i, i = 1, 2, ..., k, equals 


n! 
Ny! Ng! +++ ny! 


Theorem 4 can be proved using the product rule. We leave the details as Exercise 49. It can also 
be proved (see Exercise 50) by setting up a one-to-one correspondence between the permutations 
counted by Theorem 3 and the ways to distribute objects counted by Theorem 4. 


INDISTINGUISHABLE OBJECTS AND DISTINGUISHABLE BOXES Counting the num- 
ber of ways of placing n indistinguishable objects into k distinguishable boxes turns out to be 
the same as counting the number of n-combinations for a set with k elements when repetitions 
are allowed. The reason behind this is that there is a one-to-one correspondence between n- 
combinations from a set with k elements when repetition is allowed and the ways to place n 
indistinguishable balls into k distinguishable boxes. To set up this correspondence, we put a 
ball in the ith bin each time the ith element of the set is included in the n-combination. 


How many ways are there to place 10 indistinguishable balls into eight distinguishable bins? 


Solution: The number of ways to place 10 indistinguishable balls into eight bins equals the num- 
ber of 10-combinations from a set with eight elements when repetition is allowed. Consequently, 
there are 


17! 
C(8 + 10 — 1, 10) = C17, 10) = —— = 19,448. 
as ( 10!7! 


This means that there are C(n + r — 1, n — 1) ways to place r indistinguishable objects into 
n distinguishable boxes. 


DISTINGUISHABLE OBJECTS AND INDISTINGUISHABLE BOXES Counting the ways 
to place n distinguishable objects into k indistinguishable boxes is more difficult than count- 
ing the ways to place objects, distinguishable or indistinguishable objects, into distinguishable 
boxes. We illustrate this with an example. 
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EXAMPLE 10 How many ways are there to put four different employees into three indistinguishable offices, 
when each office can contain any number of employees? 


Solution: We will solve this problem by enumerating all the ways these employees can be placed 
into the offices. We represent the four employees by A, B, C, and D. First, we note that we can 
distribute employees so that all four are put into one office, three are put into one office and a 
fourth is put into a second office, two employees are put into one office and two put into a second 
office, and finally, two are put into one office, and one each put into the other two offices. Each 
way to distribute these employees to these offices can be represented by a way to partition the 
elements A, B, C, and D into disjoint subsets. 

We can put all four employees into one office in exactly one way, represented by 
{{A, B, C, D}}. We can put three employees into one office and the fourth employee into 
a different office in exactly four ways, represented by {{A, B, C}, {D}}, {{A, B, D}, {C}}, 
{{A, C, D}, {B}}, and {{B, C, D}, {A}}. We can put two employees into one office and two into 
a second office in exactly three ways, represented by {{A, B}, {C, D}}, {{A, C}, {B, D}}, and 
{{A, D}, {B, C}}. Finally, we can put two employees into one office, and one each into each of 
the remaining two offices in six ways, represented by {{A, B}, {C}, {D}}, {{A, C}, {B}, {D}}. 
{{A, D}, {B}, (C}}, ({B, C}, {A}, {D}}, ({B, D}}, {A}, (C}}, and {{C, D}, {A}, {By}. 

Counting all the possibilities, we find that there are 14 ways to put four different employees 
into three indistinguishable offices. Another way to look at this problem is to look at the num- 
ber of offices into which we put employees. Note that there are six ways to put four different 
employees into three indistinguishable offices so that no office is empty, seven ways to put four 
different employees into two indistinguishable offices so that no office is empty, and one way to 
put four employees into one office so that it is not empty. < 


There is no simple closed formula for the number of ways to distribute n distinguishable ob- 
jects into j indistinguishable boxes. However, there is a formula involving a summation, which 
we will now describe. Let S(n, j) denote the number of ways to distribute n distinguishable ob- 
jects into j indistinguishable boxes so that no box is empty. The numbers S(n, j) are called Stir- 
ling numbers of the second kind. For instance, Example 10 shows that S(4, 3) = 6, S(4, 2) = 7, 
and S(4, 1) = 1. We see that the number of ways to distribute n distinguishable objects into k 
indistinguishable boxes (where the number of boxes that are nonempty equals k, k — 1,..., 2, 
or 1) equals EL S(n, j). For instance, following the reasoning in Example 10, the number 
of ways to distribute four distinguishable objects into three indistinguishable boxes equals 
S(4, 1) + S(4, 2) + S(4, 3) = 1 +7 + 6 = 14. Using the inclusion—exclusion principle (see Sec- 
tion 8.6) it can be shown that 


j=l : 
sin.) = = Xen (‘Jo =. 
J* i20 r 


Consequently, the number of ways to distribute n distinguishable objects into k indistinguishable 
boxes equals 


i 
Y Sa) = 
j=l 


Remark: The reader may be curious about the Stirling numbers of the first kind. A combinatorial 
definition of the signless Stirling numbers of the first kind, the absolute values of the Stirling 
numbers of the first kind, can be found in the preamble to Exercise 47 in the Supplementary 
Exercises. For the definition of Stirling numbers of the first kind, for more information about 
Stirling numbers of the second kind, and to learn more about Stirling numbers of the first kind 


ko. j-l : 
Zev (i)u- iy" 


j=l 7° i=0 
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EXAMPLE 11 


Exercises 


and the relationship between Stirling numbers of the first and second kind, see combinatorics 
textbooks such as [B607], [Br99], and [RoTe05], and Chapter 6 in [MiRo91]. 


INDISTINGUISHABLE OBJECTS AND INDISTINGUISHABLE BOXES Some counting 
problems can be solved by determining the number of ways to distribute indistinguishable ob- 
jects into indistinguishable boxes. We illustrate this principle with an example. 


How many ways are there to pack six copies of the same book into four identical boxes, where 
a box can contain as many as six books? 


Solution: We will enumerate all ways to pack the books. For each way to pack the books, we will 
list the number of books in the box with the largest number of books, followed by the numbers 
of books in each box containing at least one book, in order of decreasing number of books in a 
box. The ways we can pack the books are 


3,2, 1 
31411 
2,2,2 

2, 2,1, 1. 


For example, 4, 1, 1 indicates that one box contains four books, a second box contains a single 
book, and a third box contains a single book (and the fourth box is empty). We conclude that 
there are nine allowable ways to pack the books, because we have listed them all. < 


Observe that distributing n indistinguishable objects into k indistinguishable boxes is the 
same as writing n as the sum of at most k positive integers in nonincreasing order. If a, + 
agt--+q=n, where dj, a, -> are positive integers with a, >a, > + > a, We say that 
a), dy, ..., @ is a partition of the positive integer n into j positive integers. We see that if p,(7) 
is the number of partitions of n into at most k positive integers, then there are p(n) ways to 
distribute n indistinguishable objects into k indistinguishable boxes. No simple closed formula 
exists for this number. For more information about partitions of positive integers, see [Rol 1]. 


1. In how many different ways can five elements be selected 
in order from a set with three elements when repetition is 
allowed? 

2. In how many different ways can five elements be selected 
in order from a set with five elements when repetition is 
allowed? 

3. How many strings of six letters are there? 

4. Every day a student randomly chooses a sandwich for 
lunch from a pile of wrapped sandwiches. If there are six 
kinds of sandwiches, how many different ways are there 
for the student to choose sandwiches for the seven days 
of a week if the order in which the sandwiches are chosen 
matters? 

5. How many ways are there to assign three jobs to five em- 
ployees if each employee can be given more than one job? 

6. How many ways are there to select five unordered ele- 
ments from a set with three elements when repetition is 
allowed? 


7. How many ways are there to select three unordered el- 
ements from a set with five elements when repetition is 
allowed? 

8. How many different ways are there to choose a dozen 
donuts from the 21 varieties at a donut shop? 

9. A bagel shop has onion bagels, poppy seed bagels, egg 
bagels, salty bagels, pumpernickel bagels, sesame seed 
bagels, raisin bagels, and plain bagels. How many ways 
are there to choose 
a) six bagels? 

b) a dozen bagels? 

c) two dozen bagels? 

d) a dozen bagels with at least one of each kind? 

e) a dozen bagels with at least three egg bagels and no 


more than two salty bagels? 

10. A croissant shop has plain croissants, cherry croissants, 
chocolate croissants, almond croissants, apple croissants, 
and broccoli croissants. How many ways are there to 
choose 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


a) a dozen croissants? 

b) three dozen croissants? 

c) two dozen croissants with at least two of each kind? 

d) two dozen croissants with no more than two broccoli 
croissants? 

e) two dozen croissants with at least five chocolate crois- 
sants and at least three almond croissants? 

f) two dozen croissants with at least one plain croissant, 
at least two cherry croissants, at least three choco- 
late croissants, at least one almond croissant, at least 
two apple croissants, and no more than three broccoli 
croissants? 


How many ways are there to choose eight coins from a 
piggy bank containing 100 identical pennies and 80 iden- 
tical nickels? 


How many different combinations of pennies, nickels, 
dimes, quarters, and half dollars can a piggy bank con- 
tain if it has 20 coins in it? 


A book publisher has 3000 copies of a discrete mathemat- 
ics book. How many ways are there to store these books 
in their three warehouses if the copies of the book are 
indistinguishable? 


How many solutions are there to the equation 
x +X +x +x = 17, 
where x), X2, X3, and x, are nonnegative integers? 
How many solutions are there to the equation 
X +X +X +2%4+X5 = 21, 
where x;, i = 1, 2, 3, 4,5, is a nonnegative integer such 
that 
a) x, 21? 
b) x; > 2 fori = 1, 2, 3, 4, 5? 
c) 0<x < 10? 
d) 0 <x <3, 1 <x < 4, and x, > 15? 
How many solutions are there to the equation 


Xi +X +X + X4 + Xs +X6 = 29, 


where x;, i = 1, 2, 3, 4, 5, 6, is a nonnegative integer such 
that 

a) x; > 1 fori = 1,2,3, 4, 5, 6? 

b) x, > 1, x = 2, x3 È 3, x4 2 4, x; > 5, and x, > 6? 
c) x; <5? 

d) x, < 8 and x, > 8? 

How many strings of 10 ternary digits (0, 1, or 2) are there 
that contain exactly two Os, three 1s, and five 2s? 


How many strings of 20-decimal digits are there that con- 
tain two Os, four 1s, three 2s, one 3, two 4s, three 5s, two 
7s, and three 9s? 


Suppose that a large family has 14 children, including two 
sets of identical triplets, three sets of identical twins, and 
two individual children. How many ways are there to seat 
these children in a row of chairs if the identical triplets or 
twins cannot be distinguished from one another? 


20. 


21. 


* 22. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 
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How many solutions are there to the inequality 
xı +X +x < 11, 


where x,, x2, and x, are nonnegative integers? [Hint: In- 
troduce an auxiliary variable x, such that x; +x, + x3 + 
x4 = 11.) 

A Swedish tour guide has devised a clever way for his 
clients to recognize him. He owns 13 pairs of shoes of 
the same style, customized so that each pair has a unique 
color. How many ways are there for him to choose a left 
shoe and a right shoe from these 13 pairs 


a) without restrictions and which color is on which foot 
matters? 

b) so that the colors of the left and right shoe are differ- 
ent and which color is on which foot matters? 

c) so that the colors of the left and right shoe are differ- 


ent but which color is on which foot does not matter? 
d) without restrictions, but which color is on which foot 


does not matter? 


In how many ways can an airplane pilot be scheduled for 
five days of flying in October if he cannot work on con- 
secutive days? 

How many ways are there to distribute six indistinguish- 
able balls into nine distinguishable bins? 

How many ways are there to distribute 12 indistinguish- 
able balls into six distinguishable bins? 

How many ways are there to distribute 12 distinguishable 
objects into six distinguishable boxes so that two objects 
are placed in each box? 

How many ways are there to distribute 15 distinguishable 
objects into five distinguishable boxes so that the boxes 
have one, two, three, four, and five objects in them, re- 
spectively. 

How many positive integers less than 1,000,000 have the 
sum of their digits equal to 19? 

How many positive integers less than 1,000,000 have ex- 
actly one digit equal to 9 and have a sum of digits equal 
to 13? 

There are 10 questions on a discrete mathematics final 
exam. How many ways are there to assign scores to the 
problems if the sum of the scores is 100 and each question 
is worth at least 5 points? 

Show that there are C(n +r -qi -qQ -+ — q, —Ln 
qı — % — + — q,) different unordered selections of n ob- 
jects of r different types that include at least q, objects of 
type one, q) objects of type two, ..., and q, objects of 
type r. 

How many different bit strings can be transmitted if the 
string must begin with a 1 bit, must include three ad- 
ditional 1 bits (so that a total of four 1 bits is sent), 
must include a total of 12 0 bits, and must have at least 
two 0 bits following each | bit? 

How many different strings can be made from the letters 
in MISSISSIPPI, using all the letters? 

How many different strings can be made from the letters 
in ABRACADABRA, using all the letters? 

How many different strings can be made from the letters 
in AARDVARK, using all the letters, if all three As must 
be consecutive? 
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35. 


36. 


37. 


38. 


39. 


40. 


41. 


42. 


43. 


44. 


45. 


46. 


47. 


How many different strings can be made from the letters 
in ORONO, using some or all of the letters? 


How many strings with five or more characters can be 
formed from the letters in SEERESS? 


How many strings with seven or more characters can be 
formed from the letters in EVERGREEN? 


How many different bit strings can be formed using 
six 1s and eight Os? 


A student has three mangos, two papayas, and two kiwi 
fruits. If the student eats one piece of fruit each day, and 
only the type of fruit matters, in how many different ways 
can these fruits be consumed? 


A professor packs her collection of 40 issues of a mathe- 

matics journal in four boxes with 10 issues per box. How 

many ways can she distribute the journals if 

a) each box is numbered, so that they are distinguish- 
able? 

b) the boxes are identical, so that they cannot be distin- 
guished? 

How many ways are there to travel in xyz space from the 

origin (0, 0, 0) to the point (4, 3,5) by taking steps one 

unit in the positive x direction, one unit in the positive y 

direction, or one unit in the positive z direction? (Moving 

in the negative x, y, or z direction is prohibited, so that no 

backtracking is allowed.) 


How many ways are there to travel in xyzw space from the 
origin (0, 0, 0, 0) to the point (4, 3,5, 4) by taking steps 
one unit in the positive x, positive y, positive z, or posi- 
tive w direction? 


How many ways are there to deal hands of seven cards to 
each of five players from a standard deck of 52 cards? 


In bridge, the 52 cards of a standard deck are dealt to four 
players. How many different ways are there to deal bridge 
hands to four players? 


How many ways are there to deal hands of five cards to 
each of six players from a deck containing 48 different 
cards? 


In how many ways can a dozen books be placed on four 

distinguishable shelves 

a) if the books are indistinguishable copies of the same 
title? 

b) if no two books are the same, and the positions of the 
books on the shelves matter? [Hint: Break this into 
12 tasks, placing each book separately. Start with the 
sequence 1, 2, 3, 4 to represent the shelves. Represent 
the books by b,, i = 1, 2, ..., 12. Place b, to the right 
of one of the terms in 1, 2, 3, 4. Then successively 
place by, b3, ..., and b;3.] 

How many ways can n books be placed on k distinguish- 

able shelves 

a) if the books are indistinguishable copies of the same 
title? 

b) if no two books are the same, and the positions of the 
books on the shelves matter? 


48. 


* 49, 


* 50. 


*51. 


52. 


53. 


54. 


55. 


56. 


57. 


58. 


A shelf holds 12 books in a row. How many ways are 
there to choose five books so that no two adjacent books 
are chosen? [Hint: Represent the books that are chosen by 
bars and the books not chosen by stars. Count the number 
of sequences of five bars and seven stars so that no two 
bars are adjacent. ] 


Use the product rule to prove Theorem 4, by first placing 
objects in the first box, then placing objects in the second 
box, and so on. 


Prove Theorem 4 by first setting up a one-to-one corre- 

spondence between permutations of n objects with n; in- 

distinguishable objects of type i, i = 1, 2, 3, ..., k, and the 
distributions of n objects in k boxes such that n, objects 
are placed in box i, i= 1, 2,3, ...,k and then applying 

Theorem 3. 

In this exercise we will prove Theorem 2 by set- 

ting up a one-to-one correspondence between the 

set of r-combinations with repetition allowed of S= 

{1, 2,3, ...,n} and the set of r-combinations of the set 

T= {1,2,3,...,.n+r—1}. 

a) Arrange the elements in an r-combination, with rep- 
etition allowed, of S into an increasing sequence 
xX, LX < +++ < x,. Show that the sequence formed by 
adding k —1 to the kth term is strictly increasing. 
Conclude that this sequence is made up of r distinct 
elements from T. 

b) Show that the procedure described in (a) defines 
a one-to-one correspondence between the set of r- 
combinations, with repetition allowed, of S and the r- 
combinations of T. [Hint: Show the correspondence 
can be reversed by associating to the r-combination 
{X1,Xy,...,X,} of T, with 1 <x) <x, < <x, S 
n+r-—1, the r-combination with repetition allowed 
from S, formed by subtracting k — 1 from the kth ele- 
ment. | 

c) Conclude that the number of r-combinations with 
repetition allowed from a set with n elements is C(n + 
r—-1,r). 

How many ways are there to distribute five distinguish- 

able objects into three indistinguishable boxes? 


How many ways are there to distribute six distinguishable 
objects into four indistinguishable boxes so that each of 
the boxes contains at least one object? 


How many ways are there to put five temporary employ- 
ees into four identical offices? 


How many ways are there to put six temporary employ- 
ees into four identical offices so that there is at least one 
temporary employee in each of these four offices? 


How many ways are there to distribute five indistinguish- 
able objects into three indistinguishable boxes? 


How many ways are there to distribute six indistinguish- 
able objects into four indistinguishable boxes so that each 
of the boxes contains at least one object? 

How many ways are there to pack eight identical DVDs 
into five indistinguishable boxes so that each box con- 
tains at least one DVD? 


59. 


60. 


61. 


62. 
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How many ways are there to pack nine identical DVDs * 63. Suppose that a weapons inspector must inspect each of 
into three indistinguishable boxes so that each box con- five different sites twice, visiting one site per day. The in- 
tains at least two DVDs? spector is free to select the order in which to visit these 
How many ways are there to distribute five balls into sites, but cannot visit site X, the most suspicious site, on 
seven boxes if each box must have at most one ball two consecutive days. In how many different orders can 
in it if the inspector visit these sites? 

a) both the balls and boxes are labeled? 64. How many different terms are there in the expansion of 
b) the balls are labeled, but the boxes are unlabeled? (xX, +X) + +++ + ,,)” after all terms with identical sets of 
c) the balls are unlabeled, but the boxes are labeled? exponents are added? 

d) both the balls and boxes are unlabeled? * 65. Prove the Multinomial Theorem: If n is a positive inte- 
How many ways are there to distribute five balls into three ger, then 

boxes if each box must have at least one ball in it if : 

a) both the balls and boxes are labeled? (i H a E Xn) 

b) the balls are labeled, but the boxes are unlabeled? = > Cn, n, ~., Ny) ae soo xii, 

c) the balls are unlabeled, but the boxes are labeled? mins = ong=A 

d) both the balls and boxes are unlabeled? h 

Suppose that a basketball league has 32 teams, split into N 

two conferences of 16 teams each. Each conference is C(n; n, Ny ..., n) = n! 

split into three divisions. Suppose that the North Central ™ o o m!m!enp! 

Division has five teams. Each of the teams in the North isa multinomial coeficient: 

Central Division plays four games against each of the i . 

other teams in this division, three games against each of 66. Find the expansion of (x + y + 2). 

the 11 remaining teams in the conference, and two games 67. Find the coefficient of x*y?2 in (x + y + z)!°. 

against each of the 16 teams in the other conference. In 68. How many terms are there in the expansion of 


how many different orders can the games of one of the 
teams in the North Central Division be scheduled? 


(x + y +z)? 


6.6. Generating Permutations and Combinations 


6.6.1 Introduction 


Links > 


Methods for counting various types of permutations and combinations were described in the 
previous sections of this chapter, but sometimes permutations or combinations need to be gen- 
erated, not just counted. Consider the following three problems. First, suppose that a salesper- 
son must visit six different cities. In which order should these cities be visited to minimize total 
travel time? One way to determine the best order is to determine the travel time for each of the 
6! = 720 different orders in which the cities can be visited and choose the one with the smallest 
travel time. Second, suppose we are given a Set of six positive integers and wish to find a subset 
of them that has 100 as their sum, if such a subset exists. One way to find these numbers is to 
generate all 2° = 64 subsets and check the sum of their elements. Third, suppose a laboratory 
has 95 employees. A group of 12 of these employees with a particular set of 25 skills is needed 
for a project. (Each employee can have one or more of these skills.) One way to find such a 
set of employees is to generate all sets of 12 of these employees and check whether they have 
the desired skills. These examples show that it is often necessary to generate permutations and 
combinations to solve problems. 


6.6.2 Generating Permutations 

Any set with n elements can be placed in one-to-one correspondence with the set {1, 2, 3, ..., n}. 
We can list the permutations of any set of n elements by generating the permutations of the n 
smallest positive integers and then replacing these integers with the corresponding elements. 
Many different algorithms have been developed to generate the n! permutations of this set. We 
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EXAMPLE 1 


EXAMPLE 2 


Extra 
Examples 


» 


will describe one of these that is based on the lexicographic (or dictionary) ordering of the 
set of permutations of {1, 2, 3, ..., n}. In this ordering, the permutation aja, --- a„ precedes the 
permutation of b; b, -+ b,, if for some k, with 1 < k <n, a, = bj, dy = by, ..., agı = Dy_,, and 
a, < b,. In other words, a permutation of the set of the n smallest positive integers precedes (in 
lexicographic order) a second permutation if the number in this permutation in the first position 
where the two permutations disagree is smaller than the number in that position in the second 
permutation. 


The permutation 23415 of the set {1, 2, 3, 4, 5} precedes the permutation 23514, because these 
permutations agree in the first two positions, but the number in the third position in the first 
permutation, 4, is smaller than the number in the third position in the second permutation, 5. 
Similarly, the permutation 41532 precedes 52143. < 


An algorithm for generating the permutations of {1, 2, ...,} can be based on a proce- 
dure that constructs the next permutation in lexicographic order following a given permutation 
aa ++ a,. We will show how this can be done. First, suppose that a,_, < a,. Interchange a„—1 
and a, to obtain a larger permutation. No other permutation is both larger than the original per- 
mutation and smaller than the permutation obtained by interchanging a,_, and a,,. For instance, 
the next larger permutation after 234156 is 234165. On the other hand, ifa,_, > a, then a larger 
permutation cannot be obtained by interchanging these last two terms in the permutation. Look 
at the last three integers in the permutation. If a,_, < a,_,, then the last three integers in the 
permutation can be rearranged to obtain the next largest permutation. Put the smaller of the two 
integers a,_, and a, that is greater than a,,_, in position n — 2. Then, place the remaining integer 
and a,,_» into the last two positions in increasing order. For instance, the next larger permutation 
after 234165 is 234516. 

On the other hand, if a,_, >a,_, (and a,_, > a„), then a larger permutation cannot be 
obtained by permuting the last three terms in the permutation. Based on these observations, a 
general method can be described for producing the next larger permutation in increasing order 
following a given permutation a,q, ---a,,. First, find the integers q and a4,, with a <a,, and 


G41 > G49 p> ay» 


that is, the last pair of adjacent integers in the permutation where the first integer in the pair 
is smaller than the second. Then, the next larger permutation in lexicographic order is ob- 
tained by putting in the jth position the least integer among 4,1; 4,5, ..., and a, that is greater 
than a, and listing in increasing order the rest of the integers Gy, yis +++ Ay in positions j + 1 to 
n. It is easy to see that there is no other permutation larger than the permutation a,a, + a, but 
smaller than the new permutation produced. (The verification of this fact is left as an exercise 
for the reader.) 


What is the next permutation in lexicographic order after 362541? 


Solution: The last pair of integers q and a; where a < a,; is a} = 2 and a, = 5. The least 
integer to the right of 2 that is greater than 2 in the permutation is a; = 4. Hence, 4 is placed 
in the third position. Then the integers 2, 5, and 1 are placed in order in the last three posi- 
tions, giving 125 as the last three positions of the permutation. Hence, the next permutation 
is 364125. < 


To produce the n! permutations of the integers 1, 2, 3, ..., n, begin with the smallest permu- 
tation in lexicographic order, namely, 123 --- n, and successively apply the procedure described 
for producing the next larger permutation of n! — 1 times. This yields all the permutations of 
the n smallest integers in lexicographic order. 


EXAMPLE 3 


Links > 
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Generate the permutations of the integers 1, 2, 3 in lexicographic order. 


Solution: Begin with 123. The next permutation is obtained by interchanging 3 and 2 to obtain 
132. Next, because 3 > 2 and 1 < 3, permute the three integers in 132. Put the smaller of 3 
and 2 in the first position, and then put 1 and 3 in increasing order in positions 2 and 3 to 
obtain 213. This is followed by 231, obtained by interchanging 1 and 3, because 1 < 3. The next 
larger permutation has 3 in the first position, followed by 1 and 2 in increasing order, namely, 
312. Finally, interchange 1 and 2 to obtain the last permutation, 321. We have generated the 
permutations of 1, 2, 3 in lexicographic order. They are 123, 132, 213, 231, 312, and 321. < 


Algorithm 1 displays the procedure for finding the next permutation in lexicographic order 
after a permutation that is notn n— 1 n—2 ... 2 1, whichis the largest permutation. 


ALGORITHM 1 Generating the Next Permutation in Lexicographic Order. 


procedure next permutation(d, a, ... a„: permutation of 
{1, 2, ..., n} notequaltonn—-1 ... 2 1) 
ji=n-l 
while a; > a); 
ji=j-1 
{ is the largest subscript with a; < aj,,} 
k:=n 
while a; > a; 
k:=k-1 
{a, is the smallest integer greater than a; to the right of a;} 
interchange a, and a, 
rion 
s:=j+1 
while r > s 
interchange a, and a, 
ri=r-1 
si=stl 
{this puts the tail end of the permutation after the jth position in increasing order} 
{aa ...a,, is now the next permutation} 


6.6.3 Generating Combinations 


How can we generate all the combinations of the elements of a finite set? Because a combination 
is just a subset, we can use the correspondence between subsets of {a}, do, ..., a, } and bit strings 
of length n. 

Recall that the bit string corresponding to a subset has a 1 in position k if a, is in the subset, 
and has a 0 in this position if a, is not in the subset. If all the bit strings of length n can be listed, 
then by the correspondence between subsets and bit strings, a list of all the subsets is obtained. 

Recall that a bit string of length n is also the binary expansion of an integer between 0 and 
2” — 1. The 2” bit strings can be listed in order of their increasing size as integers in their binary 
expansions. To produce all binary expansions of length n, start with the bit string 000 ... 00, with 
n zeros. Then, successively find the next expansion until the bit string 111... 11 is obtained. At 
each stage the next binary expansion is found by locating the first position from the right that is 
not a 1, then changing all the 1s to the right of this position to Os and making this first 0 (from 
the right) a 1. 
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EXAMPLE 4 


EXAMPLE 5 


Find the next bit string after 10 0010 0111. 


Solution: The first bit from the right that is not a 1 is the fourth bit from the right. Change 
this bit to a 1 and change all the following bits to Os. This produces the next larger bit string, 
10 0010 1000. < 


The procedure for producing the next larger bit string after b,_,b,_,...b,b 9 is given as 
Algorithm 2. 


ALGORITHM 2 Generating the Next Larger Bit String. 


procedure next bit string(b,_; b,_y...b, bo: bit string not equal to 11...11) 
i:=0 
while b; = 1 
b,:=0 
i:=it+l 
b,:=1 
{b,,_; bn-2---b1bo is now the next bit string} 


Next, an algorithm for generating the r-combinations of the set {1, 2,3, ...,n} will be 
given. An r-combination can be represented by a sequence containing the elements in the 
subset in increasing order. The r-combinations can be listed using lexicographic order on 
these sequences. In this lexicographic ordering, the first r-combination is {1,2,...,7—1,r} 
and the last r-combination is {n-—r+l1,n—r+2,...,n—1,n}. The next r-combination 
after aja, +++ a, can be obtained in the following way: First, locate the last element a; in 
the sequence such that a; # n — r + i. Then, replace a; with a; + 1 and q with a4,+j—i+1, 
for j=i+1,i+2,...,r. It is left for the reader to show that this produces the next larger 
r-combination in lexicographic order. This procedure is illustrated with Example 5. 


Find the next larger 4-combination of the set {1, 2, 3, 4,5, 6} after {1, 2, 5, 6}. 
Solution: The last term among the terms a; with a, = 1, a, = 2, a, = 5, and a, = 6 such that 
da; #6—4+iis a, = 2. To obtain the next larger 4-combination, increment a, by 1 to obtain 


a, = 3. Then set a, = 3 + 1 = 4 and a, = 3 + 2 = 5. Hence the next larger 4-combination is 
{1, 3, 4, 5}. < 


Algorithm 3 displays pseudocode for this procedure. 


ALGORITHM 3 Generating the Next r-Combination in Lexicographic Order. 


procedure next r-combination({a,, dy, ..., a}: proper subset of 
{1, 2, ..., n} not equal to {n — r + 1, ..., n} with 
a; <a, < <a) 
i:=r 
while a, =n-r+i 
i:=i-1 


ad; i=a;+1 
for j:=i+1tor 
a; :=a;+j—i 


j 
{{a;, a, ...,a,} is now the next combination } 


Exercises 


1. Place these permutations of {1, 2,3,4,5} in lexico- 
graphic order: 43521, 15432, 45321, 23451, 23514, 
14532, 21345, 45213, 31452, 31542. 

2. Place these permutations of {1,2,3,4,5,6} in lexi- 
cographic order: 234561, 231456, 165432, 156423, 
543216, 541236, 231465, 314562, 432561, 654321, 
654312, 435612. 

3. The name of a file in a computer directory consists of 
three uppercase letters followed by a digit, where each 
letter is either A, B, or C, and each digit is either 1 or 2. 
List the name of these files in lexicographic order, where 
we order letters using the usual alphabetic order of letters. 

4. Suppose that the name of a file in a computer directory 
consists of three digits followed by two lowercase letters 
and each digit is 0, 1, or 2, and each letter is either a or b. 
List the name of these files in lexicographic order, where 
we order letters using the usual alphabetic order of letters. 

5. Find the next larger permutation in lexicographic order 
after each of these permutations. 

a) 1432 b) 54123 c) 12453 
d) 45231 e) 6714235 f) 31528764 

6. Find the next larger permutation in lexicographic order 

after each of these permutations. 
a) 1342 b) 45321 c) 13245 
d) 612345 e) 1623547 f) 23587416 

7. Use Algorithm 1 to generate the 24 permutations of the 
first four positive integers in lexicographic order. 

8. Use Algorithm 2 to list all the subsets of the set 
{1, 2, 3, 4}. 

9. Use Algorithm 3 to list all the 3-combinations of 
{1, 2, 3, 4, 5}. 

10. Show that Algorithm 1 produces the next larger permu- 
tation in lexicographic order. 

11. Show that Algorithm 3 produces the next larger 
r-combination in lexicographic order after a given 
r-combination. 


Key Terms and Results 


TERMS 


combinatorics: the study of arrangements of objects 
enumeration: the counting of arrangements of objects 


tree diagram: a diagram made up of a root, branches leaving 
the root, and other branches leaving some of the endpoints 
of branches 


permutation: an ordered arrangement of the elements of a set 
r-permutation: an ordered arrangement of r elements of a set 
P(n,r): the number of r-permutations of a set with n elements 
r-combination: an unordered selection of r elements of a set 


C(n,r): the number of r-combinations of a set with n elements 
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12. Develop an algorithm for generating the r-permutations 
of a set of n elements. 


13. List all 3-permutations of {1, 2, 3, 4, 5}. 

The remaining exercises in this section develop another algo- 
rithm for generating the permutations of {1, 2, 3, ..., n}. This 
algorithm is based on Cantor expansions of integers. Every 
nonnegative integer less than n! has a unique Cantor expan- 
sion 


al! +a,2! +++ +a, i(n- 1)! 


where a; is a nonnegative integer not exceeding i, for i = 
1,2, ...,n— 1. The integers a}, aū, ..., a„_ı are called the 
Cantor digits of this integer. 

Given a permutation of {1,2,...,n}, let ap k= 
2,3, ...,n, be the number of integers less than k that fol- 
low k in the permutation. For instance, in the permutation 
43215, a, is the number of integers less than 2 that fol- 
low 2, so a, = 1. Similarly, for this example a, = 2, a, = 3, 
and a, = 0. Consider the function from the set of permu- 
tations of {1, 2, 3, ...,n} to the set of nonnegative integers 
less than n! that sends a permutation to the integer that has 
1, 4, ..., 4,1, defined in this way, as its Cantor digits. 

14. Find the Cantor digits a,, dy, ... 
these permutations. 

a) 246531 b) 12345 c) 654321 

*15. Show that the correspondence described in the pream- 
ble is a bijection between the set of permutations of 

{1, 2, 3, ..., n} and the nonnegative integers less than n!. 


, a„—ı that correspond to 


16. Find the permutations of {1, 2,3,4,5} that correspond 
to these integers with respect to the correspondence be- 
tween Cantor expansions and permutations as described 
in the preamble to Exercise 14. 

a) 3 b) 89 c) 111 

17. Develop an algorithm for producing all permutations of a 
set of n elements based on the correspondence described 
in the preamble to Exercise 14. 


binomial coefficient ( ) : also the number of r-combinations 
r 


of a set with n elements 


combinatorial proof: a proof that uses counting arguments 
rather than algebraic manipulation to prove a result 


Pascal’s triangle: a representation of the binomial coeffi- 
L 


cients where the ith row of the triangle contains () for 
J20 22807 i 


S(n, j): the Stirling number of the second kind denoting the 
number of ways to distribute n distinguishable objects into 
j indistinguishable boxes so that no box is empty 
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RESULTS 


product rule for counting: The number of ways to do a proce- 
dure that consists of two tasks is the product of the number 
of ways to do the first task and the number of ways to do 
the second task after the first task has been done. 

product rule for sets: The number of elements in the Carte- 
sian product of finite sets is the product of the number of 
elements in each set. 

sum rule for counting: The number of ways to do a task in 
one of two ways is the sum of the number of ways to do 
these tasks if they cannot be done simultaneously. 

sum rule for sets: The number of elements in the union of 
pairwise disjoint finite sets is the sum of the numbers of 
elements in these sets. 

subtraction rule for counting or inclusion-exclusion for 
sets: If a task can be done in either n} ways or n, ways, 
then the number of ways to do the task is n, + n, minus the 
number of ways to do the task that are common to the two 
different ways. 

subtraction rule or inclusion-exclusion for sets: The num- 
ber of elements in the union of two sets is the sum of the 
number of elements in these sets minus the number of ele- 
ments in their intersection. 

division rule for counting: There are n/d ways to do a task 
if it can be done using a procedure that can be carried out 
in n ways, and for every way w, exactly d of the n ways 
correspond to way w. 


Review Questions 


1. Explain how the sum and product rules can be used to 
find the number of bit strings with a length not exceed- 
ing 10. 

2. Explain how to find the number of bit strings of length 
not exceeding 10 that have at least one 0 bit. 


3. a) How can the product rule be used to find the number 

of functions from a set with m elements to a set with 
n elements? 

b) How many functions are there from a set with five el- 
ements to a set with 10 elements? 

c) How can the product rule be used to find the number 
of one-to-one functions from a set with m elements to 
a set with n elements? 

d) How many one-to-one functions are there from a set 
with five elements to a set with 10 elements? 

e) How many onto functions are there from a set with 
five elements to a set with 10 elements? 


4. How can you find the number of possible outcomes of a 
playoff between two teams where the first team that wins 
four games wins the playoff? 


5. How can you find the number of bit strings of length ten 
that either begin with 101 or end with 010? 


6. a) State the pigeonhole principle. 


division rule for sets: Suppose that a finite set A is the union 
of n disjoint subsets each with d elements. Then n = |A|/d. 


the pigeonhole principle: When more than k objects are 
placed in k boxes, there must be a box containing more than 
one object. 

the generalized pigeonhole principle: When N objects are 
placed in k boxes, there must be a box containing at least 
[N/k] objects. 


! 
Pin) = (n z r)! 


n n! 
C(n, r) = C) = rin—n! 


. sari [AFIN _ n n 
Pascal’s identity: ("7") = (,”,) + (7) 
. . y n on -k.k 
the binomial theorem: (x + y)” = X; _o e y 
There are n” r-permutations of a set with n elements when rep- 


etition is allowed. 


There are C(n +r — 1, r) r-combinations of a set with n ele- 
ments when repetition is allowed. 


There are n!/(n;!n,! + n!) permutations of n objects of k 
types where there are n; indistinguishable objects of type 
i for i = 1, 2,3, ...,k. 

the algorithm for generating the permutations of the set 
Zon} 


b) Explain how the pigeonhole principle can be used to 
show that among any 11 integers, at least two must 
have the same last digit. 


7. a) State the generalized pigeonhole principle. 
b) Explain how the generalized pigeonhole principle can 
be used to show that among any 91 integers, there are 
at least ten that end with the same digit. 


8. a) What is the difference between an r-combination and 
an r-permutation of a set with n elements? 

b) Derive an equation that relates the number of r-com- 
binations and the number of r-permutations of a set 
with n elements. 

c) How many ways are there to select six students from 
a class of 25 to serve on a committee? 

d) How many ways are there to select six students from 
a class of 25 to hold six different executive positions 
on a committee? 

9. a) What is Pascal’s triangle? 

b) How can a row of Pascal’s triangle be produced from 
the one above it? 

10. What is meant by a combinatorial proof of an identity? 

How is such a proof different from an algebraic one? 


11. Explain how to prove Pascal’s identity using a combina- 
torial argument. 


12. 


13. 


14. 


a) State the binomial theorem. 

b) Explain how to prove the binomial theorem using a 
combinatorial argument. 

c) Find the coefficient of x!0°y!0! 


(2x + 5y). 


a) Explain how to find a formula for the number of ways 
to select r objects from n objects when repetition is 
allowed and order does not matter. 

b) How many ways are there to select a dozen objects 
from among objects of five different types if objects 
of the same type are indistinguishable? 

c) How many ways are there to select a dozen objects 
from these five different types if there must be at least 
three objects of the first type? 

d) How many ways are there to select a dozen objects 
from these five different types if there cannot be more 
than four objects of the first type? 

e) How many ways are there to select a dozen objects 
from these five different types if there must be at least 
two objects of the first type, but no more than three 
objects of the second type? 


in the expansion of 


a) Letn andr be positive integers. Explain why the num- 
ber of solutions of the equation x, +x) +: +x, =T, 


Supplementary Exercises 


1. 


How many ways are there to choose 6 items from 10 dis- 

tinct items when 

a) the items in the choices are ordered and repetition is 
not allowed? 

b) the items in the choices are ordered and repetition is 
allowed? 


c) the items in the choices are unordered and repetition 
is not allowed? 


d) the items in the choices are unordered and repetition 
is allowed? 


. How many ways are there to choose 10 items from 6 dis- 


tinct items when 

a) the items in the choices are ordered and repetition is 
not allowed? 

b) the items in the choices are ordered and repetition is 
allowed? 

c) the items in the choices are unordered and repetition 
is not allowed? 


d) the items in the choices are unordered and repetition 
is allowed? 


. A test contains 100 true/false questions. How many dif- 


ferent ways can a student answer the questions on the test, 
if answers may be left blank? 


. How many strings of length 10 either start with 000 or 


end with 1111? 


. How many bit strings of length 10 over the alphabet 


{a, b, c} have either exactly three as or exactly four bs? 


15. 


16. 


17. 


18. 
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where x; is a nonnegative integer for i = 1, 2,3, ..., n, 
equals the number of r-combinations of a set with n 
elements. 

b) How many solutions in nonnegative integers are there 
to the equation x, + x) + x3 + x4 = 17? 

c) How many solutions in positive integers are there to 
the equation in part (b)? 

a) Derive a formula for the number of permutations of n 
objects of k different types, where there are n; indis- 
tinguishable objects of type one, n, indistinguishable 
objects of type two, ..., and n, indistinguishable ob- 
jects of type k. 

b) How many ways are there to order the letters of the 
word INDISCREETNESS? 


Describe an algorithm for generating all the permutations 
of the set of the smallest positive integers. 


a) How many ways are there to deal hands of five cards 
to six players from a standard 52-card deck? 

b) How many ways are there to distribute n distinguish- 
able objects into k distinguishable boxes so that n; ob- 
jects are placed in box i? 


Describe an algorithm for generating all the combinations 
of the set of the n smallest positive integers. 


. The internal telephone numbers in the phone system ona 


campus consist of five digits, with the first digit not equal 
to zero. How many different numbers can be assigned in 
this system? 


. An ice cream parlor has 28 different flavors, 8 different 


kinds of sauce, and 12 toppings. 

a) In how many different ways can a dish of three scoops 
of ice cream be made where each flavor can be used 
more than once and the order of the scoops does not 
matter? 

b) How many different kinds of small sundaes are there 
if a small sundae contains one scoop of ice cream, a 
sauce, and a topping? 

c) How many different kinds of large sundaes are there 
if a large sundae contains three scoops of ice cream, 
where each flavor can be used more than once and 
the order of the scoops does not matter; two kinds of 
sauce, where each sauce can be used only once and 
the order of the sauces does not matter; and three top- 
pings, where each topping can be used only once and 
the order of the toppings does not matter? 


. How many positive integers less than 1000 


a) have exactly three decimal digits? 

b) have an odd number of decimal digits? 

c) have at least one decimal digit equal to 9? 

d) have no odd decimal digits? 

e) have two consecutive decimal digits equal to 5? 


f) are palindromes (that is, read the same forward and 
backward)? 
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9. 


10. 


11. 


12. 


13. 


14. 


15. 


*16. 


*17; 


18. 


19. 


20. 


21. 


When the numbers from 1 to 1000 are written out in 
decimal notation, how many of each of these digits are 
used? 

a) 0 b) 1 c) 2 d) 9 

There are 12 signs of the zodiac. How many people are 

needed to guarantee that at least six of these people have 

the same sign? 

A fortune cookie company makes 213 different fortunes. 

A student eats at a restaurant that uses fortunes from this 

company and gives each customer one fortune cookie at 

the end of a meal. What is the largest possible number 
of times that the student can eat at the restaurant without 
getting the same fortune four times? 

How many people are needed to guarantee that at least 

two were born on the same day of the week and in the 

same month (perhaps in different years)? 

Show that given any set of 10 positive integers not ex- 

ceeding 50 there exist at least two different five-element 

subsets of this set that have the same sum. 

A package of baseball cards contains 20 cards. How many 

packages must be purchased to ensure that two cards in 

these packages are identical if there are a total of 550 dif- 
ferent cards? 

a) How many cards must be chosen from a standard deck 
of 52 cards to guarantee that at least two of the four 
aces are chosen? 

b) How many cards must be chosen from a standard deck 
of 52 cards to guarantee that at least two of the four 
aces and at least two of the 13 kinds are chosen? 

c) How many cards must be chosen from a standard deck 
of 52 cards to guarantee that there are at least two 
cards of the same kind? 

d) How many cards must be chosen from a standard deck 
of 52 cards to guarantee that there are at least two 
cards of each of two different kinds? 

Show that in any set of n + 1 positive integers not exceed- 

ing 2n there must be two that are relatively prime. 

Show that in a sequence of m integers there exists one or 

more consecutive terms with a sum divisible by m. 

Show that if five points are picked in the interior of a 

square with a side length of 2, then at least two of these 


points are no farther than V2 apart. 


Show that the decimal expansion of a rational number 
must repeat itself from some point onward. 

Once a computer worm infects a personal computer via an 
infected e-mail message, it sends a copy of itself to 100 e- 
mail addresses it finds in the electronic message mailbox 
on this personal computer. What is the maximum number 
of different computers this one computer can infect in the 
time it takes for the infected message to be forwarded five 
times? 

How many ways are there to choose a dozen donuts from 
20 varieties 

a) if there are no two donuts of the same variety? 

b) if all donuts are of the same variety? 

c) if there are no restrictions? 


22. 


23. 


24. 


* 25, 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


d) if there are at least two varieties among the dozen 
donuts chosen? 

e) if there must be at least six blueberry-filled donuts? 

f) if there can be no more than six blueberry-filled 
donuts? 

Find n if 

a) P(n, 2) = 110. 

c) P(n, 4) = 12P(n, 2). 

Find n if 

a) C(n, 2) = 45. 

c) C(n, 5) = C(n, 2). 

Show that if n and r are nonnegative integers and n > r, 


then 
P(n + 1,r)= P(n, rn + 1)/(n+ 1- r). 


b) P(n, n) = 5040. 


b) C(n, 3) = P(n, 2). 


Suppose that S is a set with n elements. How many or- 
dered pairs (A, B) are there such that A and B are subsets 
of S with A C B? [Hint: Show that each element of S$ be- 
longs to A, B — A, or S — B.] 


Give a combinatorial proof of Corollary 2 of Section 6.4 
by setting up a correspondence between the subsets of 
a set with an even number of elements and the subsets 
of this set with an odd number of elements. [Hint: Take 
an element a in the set. Set up the correspondence by 
putting a in the subset if it is not already in it and tak- 
ing it out if it is in the subset.] 


Let n and r be integers with 1 < r < n. Show that 
C(n,r—1)=C(n+2,r+ 1) 
—2C(n+1l,r+1)+C(nmnr+ 1). 
Prove using mathematical induction that Xa C(j, 2) = 
C(n + 1, 3) whenever n is an integer greater than 1. 
Show that if n is an integer then 
K) =4", 
2 
Show that ey x wilt (3) if n is an integer with 
n>2. 


Show that ey y 


i=1 &j=i+1 
with n > 3. 


Doa l= (") if n is an integer 


In this exercise we will derive a formula for the sum of 

the squares of the n smallest positive integers. We will 

count the number of triples (i, j, k) where i, j, and k are 

integers such that O < i < k, 0O <j < k, and 1 < k < nin 

two ways. 

a) Show that there are k? such triples with a fixed k. De- 
duce that there are $}; k? such triples. 


b) Show that the number of such triples with 
0 <i<j<k and the number of such triples with 
0 <j <i<k both equal C(n + 1, 3). 


c) Show that the number of such triples withO < i = j < 
k equals C(n + 1, 2). 


#33. 


34. 


d) Combining part (a) with parts (b) and (c), conclude 
that 


Ye = 2C(n + 1,3) + Cin + 1,2) 
= =n(n+ 1)(2n + 1)/6. 


How many bit strings of length n, where n > 4, contain 

exactly two occurrences of 01? 

Let S be a set. We say that a collection of sub- 

sets A,,A,,...,A, each containing d elements, where 

d>2, is 2-colorable if it is possible to assign to 

each element of S one of two different colors so that 

in every subset A; there are elements that have been 

assigned each color. Let m(d) be the largest integer such 

that every collection of fewer than m(d) sets each contain- 

ing d elements is 2-colorable. 

a) Show that the collection of all subsets with d elements 
of a set S with 2d — 1 elements is not 2-colorable. 

b) Show that m(2) = 3. 


**e) Show that m(3) = 7. [Hint: Show that the collec- 


35. 


36. 


37. 


38. 


39. 


40. 


41. 


tion {1, 3,5}, {1,2,6}, {1,4,7}, {2,3,4}, {2,5,7}, 
{3,6,7}, {4, 5, 6} is not 2-colorable. Then show that 
all collections of six sets with three elements each are 
2-colorable.] 

A professor writes 20 multiple-choice questions, each 

with the possible answer a, b, c, or d, for a discrete math- 

ematics test. If the number of questions with a, b, c, 

and d as their answer is 8, 3, 4, and 5, respectively, how 

many different answer keys are possible, if the questions 

can be placed in any order? 

How many different arrangements are there of eight peo- 

ple seated at a round table, where two arrangements are 

considered the same if one can be obtained from the other 

by a rotation? 

How many ways are there to assign 24 students to five 

faculty advisors? 

How many ways are there to choose a dozen apples from a 

bushel containing 20 indistinguishable Delicious apples, 

20 indistinguishable Macintosh apples, and 20 indistin- 

guishable Granny Smith apples, if at least three of each 

kind must be chosen? 

How many solutions are there to the equation x, + x) + 

x, = 17, where x,, Xx), and x, are nonnegative integers 

with 

a) x; > 1, x, > 2, and x; > 3? 

b) x; < 6 and x, > 5? 

c) x; < 4, x < 3, and x; > 5? 

a) How many different strings can be made from the 
word PEPPERCORN when all the letters are used? 

b) How many of these strings start and end with the letter P? 

c) In how many of these strings are the three letter Ps 
consecutive? 


How many subsets of a set with ten elements 


a) have fewer than five elements? 
b) have more than seven elements? 
c) have an odd number of elements? 
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42. A witness to a hit-and-run accident tells the police that 
the license plate of the car in the accident, which con- 
tains three letters followed by three digits, starts with the 
letters AS and contains both the digits 1 and 2. How many 
different license plates can fit this description? 

43. How many ways are there to put n identical objects into 
m distinct containers so that no container is empty? 

44. How many ways are there to seat six boys and eight girls 
in a row of chairs so that no two boys are seated next to 
each other? 

45. How many ways are there to distribute six objects to five 
boxes if 
a) both the objects and boxes are labeled? 

b) the objects are labeled, but the boxes are unlabeled? 
c) the objects are unlabeled, but the boxes are labeled? 
d) both the objects and the boxes are unlabeled? 

46. How many ways are there to distribute five objects into 
six boxes if 
a) both the objects and boxes are labeled? 

b) the objects are labeled, but the boxes are unlabeled? 
c) the objects are unlabeled, but the boxes are labeled? 
d) both the objects and the boxes are unlabeled? 

The signless Stirling number of the first kind c(n, k), 
where k and n are integers with 1 < k < n, equals the num- 
ber of ways to arrange n people around k circular tables with 
at least one person seated at each table, where two seatings 
of m people around a circular table are considered the same if 
everyone has the same left neighbor and the same right neigh- 
bor. 

47. Find these signless Stirling numbers of the first kind. 

a) c(3, 2) b) c(4, 2) 
c) c(4, 3) d) c(5, 4) 

48. Show that if n is a positive integer, then Di c(n, j) =n!. 

49. Show that if n is a positive integer with n > 3, then 

c(n, n — 2) = (3n — 1)C(n, 3)/4. 

Show that if n and k are integers with 1 < k <n, then 

c(n+ 1, k) = c(n, k — 1) + nc(n, k). 

51. Give a combinatorial proof that 2” divides n! whenever n 
is an even positive integer. [Hint: Use Theorem 3 in Sec- 
tion 6.5 to count the number of permutations of 2n objects 
where there are two indistinguishable objects of n differ- 
ent types.] 


* 50. 


52. How many 11-element RNA sequences consist of 4 As, 
3 Cs, 2 Us, and 2 Gs, and end with CAA? 
Exercises 53 and 54 are based on a discussion in [RoTe09]. 
A method used in the 1960s for sequencing RNA chains used 
enzymes to break chains after certain links. Some enzymes 
break RNA chains after each G link, while others break them 
after each C or U link. Using these enzymes it is sometimes 
possible to correctly sequence all the bases in an RNA chain. 
*53. Suppose that when an enzyme that breaks RNA chains af- 
ter each G link is applied to a 12-link chain, the fragments 
obtained are G, CCG, AAAG, and UCCG, and when an 
enzyme that breaks RNA chains after each C or U link 
is applied, the fragments obtained are C, C, C, C, GGU, 
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* 54. 


and GAAAG. Can you determine the entire 12-link RNA 
chain from these two sets of fragments? If so, what is this 
RNA chain? 


Suppose that when an enzyme that breaks RNA chains 
after each G link is applied to a 12-link chain, the frag- 
ments obtained are AC, UG, and ACG and when an en- 
zyme that breaks RNA chains after each C or U link is 
applied, the fragments obtained are U, GAC, and GAC. 
Can you determine the entire RNA chain from these two 
sets of fragments? If so, what is this RNA chain? 


Computer Projects 


Write programs with these input and output. 


1. Given a positive integer n and a nonnegative integer not 


*4, 


exceeding n, find the number of r-permutations and r- 
combinations of a set with n elements. 


. Given positive integers n and r, find the number of 


r-permutations when repetition is allowed and r-com- 
binations when repetition is allowed of a set with n el- 
ements. 


. Given a sequence of positive integers, find the longest in- 


creasing and the longest decreasing subsequence of the 
sequence. 


Given an equation x; + x) + + +x, = C, where C is a 
constant, and x4, Xz, ..., X, are nonnegative integers, list 
all the solutions. 


. Given a positive integer n, list all the permutations of the 


set {1, 2,3, ..., n} in lexicographic order. 


Computations and Explorations 


55. 


56. 


*57. 


*58. 


10. 


Devise an algorithm for generating all the r-permutations 
of a finite set when repetition is allowed. 

Devise an algorithm for generating all the r-combinations 
of a finite set when repetition is allowed. 

Show that if m and n are integers with m > 3 and n > 3, 
then R(m, n) < R(m, n — 1) + R(m — 1, n). 

Show that R(3, 4) > 7 by showing that in a group of six 
people, where any two people are friends or enemies, 
there are not necessarily three mutual friends or four mu- 
tual enemies. 


. Given a positive integer n and a nonnegative integer r 


not exceeding n, list all the r-combinations of the set 
{1, 2, 3, ..., n} in lexicographic order. 


. Given a positive integer n and a nonnegative integer r 


not exceeding n, list all the r-permutations of the set 
{1, 2, 3, ..., n} in lexicographic order. 


. Given a positive integer n, list all the combinations of the 


set {1, 2, 3,...,n}. 


. Given positive integers n andr, list all the r-permutations, 


with repetition allowed, of the set {1, 2, 3, ..., n}. 


Given positive integers n and r, list all the r- 
combinations, with repetition allowed, of the set 
{1, 2, 3,...,n}. 


Use a computational program or programs you have written to do these exercises. 


. Find the number of possible outcomes in a two-team play- 


off when the winner is the first team to win 5 out of 9, 6 
out of 11, 7 out of 13, and 8 out of 15. 


. Which binomial coefficients are odd? Can you formulate a 


conjecture based on numerical evidence? 


. Verify that C(2n, n) is divisible by the square of a prime, 


when n Æ 1,2, or 4, for as many positive integers n as 
you can. [That C(2n, n) is divisible by the square of a 
prime with n # 1, 2, or 4 was proved in 1996 by Andrew 
Granville and Olivier Ramaré, settling a conjecture made 
in 1980 by Paul Erdős and Ron Graham. ] 


. Find as many odd integers n less than 200 as you can for 


which C(n, |n/2]) is not divisible by the square of a prime. 
Formulate a conjecture based on your evidence. 


ES, 


For each integer less than 100 determine whether C(2n, n) 
is divisible by 3. Can you formulate a conjecture that tells 
us for which integers n the binomial coefficient C(2n, n) is 
divisible by 3 based on the digits in the base three expan- 
sion of n? 


. Generate all the permutations of a set with eight elements. 


. Generate all the 6-permutations of a set with nine ele- 


ments. 


. Generate all combinations of a set with eight elements. 


. Generate all 5-combinations with repetition allowed of a 


set with seven elements. 


Writing Projects 


Respond to these with essays using outside sources. 


. Describe some of the earliest uses of the pigeonhole prin- 
ciple by Dirichlet and other mathematicians. 


. Discuss ways in which the current telephone numbering 
plan can be extended to accommodate the rapid demand 
for more telephone numbers. (See if you can find some 
of the proposals coming from the telecommunications in- 
dustry.) For each new numbering plan you discuss, show 
how to find the number of different telephone numbers it 
supports. 


. Discuss the importance of combinatorial reasoning in gene 
sequencing and related problems involving genomes. 


. Many combinatorial identities are described in this book. 
Find some sources of such identities and describe im- 
portant combinatorial identities besides those already in- 
troduced in this book. Give some representative proofs, 
including combinatorial ones, of some of these identities. 


. Describe the different models used to model the dis- 
tribution of particles in statistical mechanics, including 


10. 
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Maxwell—Boltzmann, Bose—Einstein, and Fermi—Dirac 
statistics. In each case, describe the counting techniques 
used in the model. 


. Define the Stirling numbers of the first kind and describe 


some of their properties and the identities they satisfy. 


. Describe some of the properties and the identities that 


Stirling numbers of the second kind satisfy, including the 
connection between Stirling numbers of the first and sec- 
ond kinds. 


. Describe the latest discoveries of values and bounds for 


Ramsey numbers. 


. Describe additional ways to generate all the permutations 


of a set with n elements besides those found in Section 
6.6. Compare these algorithms and the algorithms de- 
scribed in the text and exercises of Section 6.6 in terms 
of their computational complexity. 

Describe at least one way to generate all the partitions of 
a positive integer n. (See Exercise 49 in Section 5.3.) 


7.1 An 
Introduction to 
Discrete 
Probability 


7.2 Probability 
Theory 


7.3 Bayes’ 
Theorem 


7.4 Expected Value 
and Variance 


Discrete Probability 


C ombinatorics and probability theory share common origins. The theory of probability was 
first developed more than 300 years ago, when certain gambling games were analyzed. 
Although probability theory was originally invented to study gambling, it now plays an essential 
role in a wide variety of disciplines. For example, probability theory is extensively applied in the 
study of genetics, where it can be used to help understand the inheritance of traits. Probability 
theory remains popular both because of its applicability to gambling, an extremely popular 
human endeavor, and to its use in a wide range of disciplines. 

In computer science, probability theory plays an important role in the study of the com- 
plexity of algorithms. In particular, ideas and techniques from probability theory are used to 
determine the average-case complexity of algorithms. Probabilistic algorithms can be used to 
solve many problems that cannot be easily or practically solved by deterministic algorithms. In 
a probabilistic algorithm, instead of always following the same steps when given the same in- 
put, as a deterministic algorithm does, the algorithm makes one or more random choices, which 
may lead to different output. In combinatorics, probability theory can even be used to show that 
objects with certain properties exist. The probabilistic method, a technique in combinatorics 
introduced by Paul Erdős and Alfréd Rényi, shows that an object with a specified property ex- 
ists by showing that there is a positive probability that a randomly constructed object has this 
property. Probability theory can help us answer questions that involve uncertainty, such as de- 
termining whether we should reject an incoming mail message as spam based on the words that 
appear in the message. 


An Introduction to Discrete Probability 


7.1.1 Introduction 


Probability theory dates back to 1526 when the Italian mathematician, physician, and gam- 
bler Girolamo Cardano wrote the first known systematic treatment of the subject in his book 
Liber de Ludo Aleae (Book on Games of Chance). (This book was not published until 1663, 
which may have held back the development of probability theory.) In the seventeenth century 
the French mathematician Blaise Pascal determined the odds of winning some popular bets 
based on the outcome when a pair of dice is repeatedly rolled. In the eighteenth century, the 
French mathematician Laplace, who also studied gambling, defined the probability of an event 
as the number of successful outcomes divided by the number of possible outcomes. For instance, 
the probability that a die comes up an odd number when it is rolled is the number of successful 
outcomes—namely, the number of ways it can come up odd—divided by the number of pos- 
sible outcomes—namely, the number of different ways the die can come up. There are a total 
of six possible outcomes—namely, 1, 2, 3, 4, 5, and 6—and exactly three of these are successful 
outcomes—namely, 1, 3, and 5. Hence, the probability that the die comes up an odd number is 
3/6 = 1/2. (Note that it has been assumed that all possible outcomes are equally likely, or, in 
other words, that the die is fair.) 

In this section we will restrict ourselves to experiments that have finitely many, equally 
likely, outcomes. This permits us to use Laplace’s definition of the probability of an event. We 
will continue our study of probability in Section 7.2, where we will study experiments with 
finitely many outcomes that are not necessarily equally likely. In Section 7.2 we will also intro- 
duce some key concepts in probability theory, including conditional probability, independence 
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of events, and random variables. In Section 7.4 we will introduce the concepts of the expectation 
and variance of a random variable. 


7.1.2 Finite Probability 


An experiment is a procedure that yields one of a given set of possible outcomes. The sample 
space of the experiment is the set of possible outcomes. An event is a subset of the sample 
space. Laplace’s definition of the probability of an event with finitely many possible outcomes 
will now be stated. 


Definition 1 If S is a finite nonempty sample space of equally likely outcomes, and E is an event, that 


E 
is, a subset of S, then the probability of E is p(E) = a 
The probability of an According to Laplace’s definition, the probability of an event is between 0 and 1. To see this, 


event can never be 
negative or more than 
one! 


note that if E is an event from a finite sample space S, then 0 < |E| < |S], because E C S. Thus, 
0 < p(E) = |El/|S| < 1. 
Examples 1-7 illustrate how the probability of an event is found. 


EXAMPLE 1 An urn contains four blue balls and five red balls. What is the probability that a ball chosen at 
random from the urn is blue? 


Extra » Solution: To calculate the probability, note that there are nine possible outcomes, and four of 
Examples these possible outcomes produce a blue ball. Hence, the probability that a blue ball is chosen 
is 4/9. < 


EXAMPLE 2 What is the probability that when two dice are rolled, the sum of the numbers on the two dice 
is 7? 


Solution: There are a total of 36 equally likely possible outcomes when two dice are rolled. 
(The product rule can be used to see this; because each die has six possible outcomes, the total 
number of outcomes when two dice are rolled is 67 = 36.) There are six successful outcomes, 
namely, (1, 6), (2, 5), (3, 4), (4, 3), (5, 2), and (6, 1), where the values of the first and second dice 


GIROLAMO CARDANO (1501-1576) (Cardano, born in Pavia, Italy, was the illegitimate child of Fazio 
Cardano, a lawyer, mathematician, and friend of Leonardo da Vinci, and Chiara Micheria, a young widow. 
In spite of illness and poverty, Cardano was able to study at the universities of Pavia and Padua, from where 
he received his medical degree. Cardano was not accepted into Milan’s College of Physicians because of his 
illegitimate birth, as well as his eccentricity and confrontational style. Nevertheless, his medical skills were 
highly regarded. One of his main accomplishments as a physician is the first description of typhoid fever. 

Cardano published more than 100 books on a diverse range of subjects, including medicine, the nat- 
ural sciences, mathematics, gambling, physical inventions and experiments, and astrology. He also wrote a 
@Science History fascinating autobiography. In mathematics, Cardano’s book Ars Magna, published in 1545, established the 
Images/Alamy Stock Photo foundations of abstract algebra. This was the most comprehensive book on abstract algebra for more than a 

century; it presents many novel ideas of Cardano and of others, including methods for solving cubic and quartic 
equations from their coefficients. Cardano also made several important contributions to cryptography. Cardano was an advocate of 
education for the deaf, believing, unlike his contemporaries, that deaf people could learn to read and write before learning to speak, 
and could use their minds just as well as hearing people. 

Cardano was often short of money. However, he kept himself solvent through gambling and winning money by beating others 
at chess. His book about games of chance, Liber de Ludo Aleae, written in 1526 (but published in 1663), offers the first systematic 
treatment of probability; it also describes effective ways to cheat. Cardano was considered to be a man of dubious moral character; 
he was often described as a liar, gambler, lecher, and heretic. 
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are represented by an ordered pair. Hence, the probability that a seven comes up when two fair 
dice are rolled is 6/36 = 1/6. < 


Lotteries are extremely popular throughout the world. We can easily compute the odds of 
winning different types of lotteries, as illustrated in Examples 3 and 4. (The odds of winning 
prizes in the popular Mega Millions and Powerball lotteries, as of 2018, are studied in Exercises 
38-41. The rules for these games change every few years. The odds for the version of these 
games in 2012 are studied in the Supplementary Exercises.) 


In a lottery, players win a large prize when they pick four digits that match, in the correct order, 
four digits selected by a random mechanical process. A smaller prize is won if only three digits 
are matched. What is the probability that a player wins the large prize? What is the probability 
that a player wins the small prize? 


Solution: There is only one way to choose all four digits correctly. By the product rule, there 
are 10* = 10,000 ways to choose four digits. Hence, the probability that a player wins the large 
prize is 1/10,000 = 0.0001. 

Players win the smaller prize when they correctly choose exactly three of the four digits. 
Exactly one digit must be wrong to get three digits correct, but not all four correct. By the sum 
tule, to find the number of ways to choose exactly three digits correctly, we add the number of 
ways to choose four digits matching the digits picked in all but the ith position, for i = 1, 2, 3, 4. 

To count the number of successes with the first digit incorrect, note that there are nine 
possible choices for the first digit (all but the one correct digit), and one choice for each of the 
other digits, namely, the correct digits for these slots. Hence, there are nine ways to choose four 
digits where the first digit is incorrect, but the last three are correct. Similarly, there are nine 
ways to choose four digits where the second digit is incorrect, nine with the third digit incorrect, 
and nine with the fourth digit incorrect. Hence, there is a total of 36 ways to choose four digits 
with exactly three of the four digits correct. Thus, the probability that a player wins the smaller 
prize is 36/10,000 = 9/2500 = 0.0036. < 


There are many lotteries now that award enormous prizes to people who correctly choose a set 
of six numbers out of the first n positive integers, where n is usually between 30 and 60. What 
is the probability that a person picks the correct six numbers out of 40? 


Solution: There is only one winning combination. The total number of ways to choose six num- 
bers out of 40 is 


40! 


= 3,838,380. 


PIERRE-SIMON LAPLACE (1749-1827) Pierre-Simon Laplace came from humble origins in Normandy. 
In his childhood he was educated in a school run by the Benedictines. At 16 he entered the University of Caen 
intending to study theology. However, he soon realized his true interests were in mathematics. After completing 
his studies, he was named a provisional professor at Caen, and in 1769 he became professor of mathematics at 
the Paris Military School. 

Laplace is best known for his contributions to celestial mechanics, the study of the motions of heavenly 
bodies. His Traité de Mécanique Céleste is considered one of the greatest scientific works of the early nineteenth 
century. Laplace was one of the founders of probability theory and made many contributions to mathematical 
statistics. His work in this area is documented in his book Théorie Analytique des Probabilités, in which he 
defined the probability of an event as the ratio of the number of favorable outcomes to the total number of 
outcomes of an experiment. 


Laplace was famous for his political flexibility. He was loyal, in succession, to the French Republic, Napoleon, and King Louis 
XVIII. This flexibility permitted him to be productive before, during, and after the French Revolution. 
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Consequently, the probability of picking a winning combination is 1 /3,838,380 ~ 0.00000026. 
(Here the symbol ~ means approximately equal to.) < 


Poker, and other card games, are growing in popularity. To win at these games it helps to 
know the probability of different hands. We can find the probability of specific hands that arise 
in card games using the techniques developed so far. A deck of cards contains 52 cards. There 
are 13 different kinds of cards, with four cards of each kind. (Among the terms commonly used 
instead of “kind” are “rank,” “face value,” “denomination,” and “value.”) These kinds are twos, 
threes, fours, fives, sixes, sevens, eights, nines, tens, jacks, queens, kings, and aces. There are 
also four suits: spades, clubs, hearts, and diamonds, each containing 13 cards, with one card of 
each kind in a suit. In many poker games, a hand consists of five cards. 


Find the probability that a hand of five cards in poker contains four cards of one kind. 


Solution: By the product rule, the number of hands of five cards with four cards of one kind 
is the product of the number of ways to pick one kind, the number of ways to pick the four of 
this kind out of the four in the deck of this kind, and the number of ways to pick the fifth card. 
This is 


C13, C4, 4)C(48, 1). 


By Example 11 in Section 6.3 there are C(52, 5) different hands of five cards. Hence, the 
probability that a hand contains four cards of one kind is 


C13, DC4, 4)C(48,1) 13-1-48 
FEF eo 7 0.00024. 
C(52, 5) 2,598,960 < 


What is the probability that a poker hand contains a full house, that is, three of one kind and 
two of another kind? 


Solution: By the product rule, the number of hands containing a full house is the product of 
the number of ways to pick two kinds in order, the number of ways to pick three out of four 
for the first kind, and the number of ways to pick two out of four for the second kind. (Note that 
the order of the two kinds matters, because, for instance, three queens and two aces is different 
from three aces and two queens.) We see that the number of hands containing a full house is 


P(13, 2)C(4, 3)C(4, 2) = 13-12 - 4-6 = 3744. 


Because there are C(52, 5) = 2,598,960 poker hands, the probability of a full house is 


3744 
a 00014, 
2,598,960 < 


What is the probability that the numbers 11, 4, 17, 39, and 23 are drawn in that order from a bin 
containing 50 balls labeled with the numbers 1, 2, ..., 50 if (a) the ball selected is not returned 
to the bin before the next ball is selected and (b) the ball selected is returned to the bin before 
the next ball is selected? 


Solution: (a) By the product rule, there are 50 - 49 - 48 - 47 - 46 = 254,251,200 ways to select 
the balls because each time a ball is drawn there is one fewer ball to choose from. Consequently, 


THEOREM 1 


EXAMPLE 8 


THEOREM 2 
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the probability that 11, 4, 17, 39, and 23 are drawn in that order is 1/254,251,200. This is an 
example of sampling without replacement. 


(b) By the product rule, there are 50° = 312,500,000 ways to select the balls because there are 
50 possible balls to choose from each time a ball is drawn. Consequently, the probability that 
11, 4, 17, 39, and 23 are drawn in that order is 1/312,500,000. This is an example of sampling 
with replacement. < 


7.1.3 Probabilities of Complements and Unions of Events 


We can use counting techniques to find the probability of events derived from other events. 


Let E be an event in a sample space S. The probability of the event E = S — E, the comple- 
mentary event of E, is given by 


p(E) = 1 — p(E). 


Proof: To find the probability of the event E = S — E, note that |E] = |S| — |E]. Hence, 


—  |S|-|E| |E| 
E) = =—— =1- Se le lh, 
p(E) is] is| D(E) < 


There is an alternative strategy for finding the probability of an event when a direct approach 
does not work well. Instead of determining the probability of the event, the probability of its 
complement can be found. This is often easier to do, as Example 8 shows. 


A sequence of 10 bits is randomly generated. What is the probability that at least one of these 
bits is 0? 


Solution: Let E be the event that at least one of the 10 bits is 0. Then F is the event that all the 
bits are 1s. Because the sample space S is the set of all bit strings of length 10, it follows that 


= JE] 1 

E)=1-p(é)=1-— =1-— 

p(E) p(E) S| 510 
-ıl _ 1023 
1024  1024` 


Hence, the probability that the bit string will contain at least one 0 bit is 1023/1024. It is quite 
difficult to find this probability directly without using Theorem 1. 


We can also find the probability of the union of two events. 


Let E, and E, be events in the sample space S. Then 


P(E, U E3) = p(E,) + p(E,) — p(E, A B). 
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Proof: Using the formula given in Section 2.2 for the number of elements in the union of two 
sets, it follows that 


|E, UE,| = |E,| + |E] — |E; n El. 


Hence, 
evn |E, UE,| 
_ IE, | + lE- |EN E| 
S| 
_lEl lApl | IE, n=l 
IS] S| S| 
= p(E,) + pE?) — p(E; N Ey). < 


What is the probability that a positive integer selected at random from the set of positive integers 
not exceeding 100 is divisible by either 2 or 5? 


Solution: Let E, be the event that the integer selected at random is divisible by 2, and let E, be 
the event that it is divisible by 5. Then E; U E, is the event that it is divisible by either 2 or 5. 
Also, E, N E, is the event that it is divisible by both 2 and 5, or equivalently, that it is divisible 
by 10. Because |E; | = 50, |E,| = 20, and |E, N E| = 10, it follows that 


P(E, U Ey) = p(E,) + p(E,) — pE NE) 


_ 50, 20 10 _3 
100 100 100 5° < 


7.1.4 Probabilistic Reasoning 


A common problem is determining which of two events is more likely. Analyzing the probabil- 
ities of such events can be tricky. Example 10 describes a problem of this type. It discusses a 
famous problem originating with the television game show Let’s Make a Deal and named after 
the host of the show, Monty Hall. 


The Monty Hall Three-Door Puzzle Suppose you are a game show contestant. You have a 
chance to win a large prize. You are asked to select one of three doors to open; the large prize 
is behind one of the three doors and the other two doors are losers. Once you select a door, the 
game show host, who knows what is behind each door, does the following. First, whether or not 
you selected the winning door, he opens one of the other two doors that he knows is a losing 
door (selecting at random if both are losing doors). Then he asks you whether you would like to 
switch doors. Which strategy should you use? Should you change doors or keep your original 
selection, or does it not matter? 


Solution: The probability you select the correct door (before the host opens a door and asks you 
whether you want to change) is 1/3, because the three doors are equally likely to be the correct 
door. The probability this is the correct door does not change once the game show host opens 
one of the other doors, because he will always open a door that the prize is not behind. 

The probability that you selected incorrectly is the probability the prize is behind one of the 
two doors you did not select. Consequently, the probability that you selected incorrectly is 2/3. 
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If you selected incorrectly, when the game show host opens a door to show you that the prize is 
not behind it, the prize is behind the other door. You will always win if your initial choice was 
incorrect and you change doors. So, by changing doors, the probability you win is 2/3. In other 
words, you should always change doors when given the chance to do so by the game show host. 
This doubles the probability that you will win. (A more rigorous treatment of this puzzle can be 
found in Exercise 15 of Section 7.3. For much more on this notorious puzzle and its variations, 


see [Ro09].) 


Exercises 


16. 


17. 


18. 


. What is the probability that a card selected at random 


from a standard deck of 52 cards is an ace? 


. What is the probability that a fair die comes up six when 


it is rolled? 


. What is the probability that a randomly selected integer 


chosen from the first 100 positive integers is odd? 


. What is the probability that a randomly selected day of a 


leap year (with 366 possible days) is in April? 


. What is the probability that the sum of the numbers on 


two dice is even when they are rolled? 


. What is the probability that a card selected at random 


from a standard deck of 52 cards is an ace or a heart? 


. What is the probability that when a coin is flipped six 


times in a row, it lands heads up every time? 


. What is the probability that a five-card poker hand con- 


tains the ace of hearts? 


. What is the probability that a five-card poker hand does 


not contain the queen of hearts? 


. What is the probability that a five-card poker hand con- 


tains the two of diamonds and the three of spades? 


. What is the probability that a five-card poker hand con- 


tains the two of diamonds, the three of spades, the six of 
hearts, the ten of clubs, and the king of hearts? 


. What is the probability that a five-card poker hand con- 


tains exactly one ace? 


. What is the probability that a five-card poker hand con- 


tains at least one ace? 


. What is the probability that a five-card poker hand con- 


tains cards of five different kinds? 


. What is the probability that a five-card poker hand con- 


tains two pairs (that is, two of each of two different kinds 
and a fifth card of a third kind)? 

What is the probability that a five-card poker hand con- 
tains a flush, that is, five cards of the same suit? 

What is the probability that a five-card poker hand con- 
tains a straight, that is, five cards that have consecutive 
kinds? (Note that an ace can be considered either the low- 
est card of an A-2-3-4-5 straight or the highest card of a 
10-J-Q-K-A straight.) 

What is the probability that a five-card poker hand con- 
tains a straight flush, that is, five cards of the same suit of 
consecutive kinds? 


*19 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


< 


. What is the probability that a five-card poker hand con- 
tains cards of five different kinds and does not contain a 
flush or a straight? 


What is the probability that a five-card poker hand con- 
tains a royal flush, that is, the 10, jack, queen, king, and 
ace of one suit? 


What is the probability that a fair die never comes up an 
even number when it is rolled six times? 


What is the probability that a positive integer not exceed- 
ing 100 selected at random is divisible by 3? 


What is the probability that a positive integer not exceed- 
ing 100 selected at random is divisible by 5 or 7? 


Find the probability of winning a lottery by selecting the 
correct six integers, where the order in which these inte- 
gers are selected does not matter, from the positive inte- 
gers not exceeding 

a) 30. b) 36. c) 42. d) 48. 

Find the probability of winning a lottery by selecting the 
correct six integers, where the order in which these inte- 
gers are selected does not matter, from the positive inte- 
gers not exceeding 

a) 50. b) 52. c) 56. d) 60. 

Find the probability of selecting none of the correct six 
integers in a lottery, where the order in which these inte- 
gers are selected does not matter, from the positive inte- 
gers not exceeding 

a) 40. b) 48. c) 56. d) 64. 

Find the probability of selecting exactly one of the correct 
six integers in a lottery, where the order in which these 
integers are selected does not matter, from the positive 
integers not exceeding 

a) 40. b) 48. c) 56. d) 64. 

In a superlottery, a player selects 7 numbers out of the 
first 80 positive integers. What is the probability that a 
person wins the grand prize by picking 7 numbers that 
are among the 11 numbers selected at random by a com- 
puter. 


In a superlottery, players win a fortune if they choose the 
eight numbers selected by a computer from the positive 
integers not exceeding 100. What is the probability that 
a player wins this superlottery? 
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30. 


31. 


32. 


33. 


34. 


35. 


36. 


37. 


38. 


What is the probability that a player of a lottery wins 
the prize offered for correctly choosing five (but not six) 
numbers out of six integers chosen at random from the 
integers between 1 and 40, inclusive? 


Suppose that 100 people enter a contest and that different 
winners are selected at random for first, second, and third 
prizes. What is the probability that Michelle wins one of 
these prizes if she is one of the contestants? 


Suppose that 100 people enter a contest and that differ- 

ent winners are selected at random for first, second, and 

third prizes. What is the probability that Kumar, Janice, 

and Pedro each win a prize if each has entered the con- 

test? 

What is the probability that Abby, Barry, and Sylvia win 

the first, second, and third prizes, respectively, in a draw- 

ing if 200 people enter a contest and 

a) no one can win more than one prize. 

b) winning more than one prize is allowed. 

What is the probability that Bo, Colleen, Jeff, and 

Rohini win the first, second, third, and fourth prizes, 

respectively, in a drawing if 50 people enter a contest and 

a) no one can win more than one prize. 

b) winning more than one prize is allowed. 

In roulette, a wheel with 38 numbers is spun. Of these, 18 

are red, and 18 are black. The other two numbers, which 

are neither black nor red, are 0 and 00. The probability 

that when the wheel is spun it lands on any particular 

number is 1/38. 

a) What is the probability that the wheel lands on a red 
number? 


b) What is the probability that the wheel lands on a black 
number twice in a row? 

c) What is the probability that the wheel lands on 0 or 
00? 

d) What is the probability that in five spins the wheel 
never lands on either 0 or 00? 

e) What is the probability that the wheel lands on one of 


the first six integers on one spin, but does not land on 
any of them on the next spin? 
Which is more likely: rolling a total of 8 when two dice 
are rolled or rolling a total of 8 when three dice are rolled? 


Which is more likely: rolling a total of 9 when two dice 
are rolled or rolling a total of 9 when three dice are rolled? 


A player in the Mega Millions lottery picks five different 

integers between 1 and 70, inclusive, and a sixth integer 

between 1 and 25, inclusive, which may duplicate one of 

the earlier five integers. The player wins the jackpot if all 

six numbers match the numbers drawn. 

a) What is the probability that a player wins the jackpot? 

b) What is the probability that a player wins $1,000,000, 
the prize for matching the first five numbers, but not 
the sixth number, drawn? 

c) What is the probability that a player wins $500, the 
prize for matching exactly four of the first five num- 
bers, but not the sixth number, drawn? 
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d) What is the probability that a player wins $10, the 
prize for matching exactly three of the first five num- 
bers but not the sixth number drawn, or for match- 
ing exactly two of the first five numbers and the sixth 
number drawn? 

When a player buys a Mega Millions ticket in many states 

(see Exercise 38), the player can also buy the Megaplier, 

which multiplies the size of a prize other than a jackpot 

by a multiplier ranging from two to five. The Megaplier 
is drawn using a pool of 15 balls, with five marked 2X, 
six marked 3X, three marked 4X, and one marked 5X, 
where each ball has the same likelihood of being drawn. 

Find the probability that a player who buys a Mega Mil- 

lions ticket and the Megaplier wins 

a) $5,000,000? (The only way to do this is to match 
the first five numbers drawn but not the sixth number 
drawn, with Megaplier 5X.) 

b) $30,000? (The only way to do this is to match exactly 
four of the first five numbers drawn and the sixth num- 
ber drawn, with Megaplier 3X.) 
$20? (The three ways to do this are to match exactly 
three of the first five numbers drawn, but not the sixth 
number drawn, or exactly two of the first five num- 
bers and the sixth number, with Megaplier 2X, or to 
match exactly one of the first five numbers and the 
sixth number, with Megaplier 5X.) 

d) $8? (The two ways to do this are to match exactly one 
of the first five numbers and the sixth number drawn, 
with a multiplier of 2X, or to match the sixth num- 
ber but none of the first five numbers, with Mega- 
plier 4X.) 

A player in the Powerball lottery picks five different in- 

tegers between 1 and 69, inclusive, and a sixth integer 

between 1 and 26, which may duplicate one of the ear- 
lier five integers. The player wins the jackpot if all six 
numbers match the numbers drawn. 


c) 


a) What is the probability that a player wins the jackpot? 

b) What is the probability that a player wins $1,000,000, 
which is the prize for matching the first five numbers, 
but not the sixth number, drawn? 

c) What is the probability that a player wins $100 by 
matching exactly three of the first five and the sixth 
numbers drawn, or four of the first five numbers, but 
not the sixth number, drawn? 


d) What is the probability that a player wins a prize of 
$4, which is the prize when the player matches the 
sixth number, and either one or none of the first five 
numbers drawn? 


A player in the Powerball lottery (see Exercise 40) can 
purchase the Power Play option. When this option has 
been purchased, prizes other than the jackpot are mul- 
tiplied by a multiplier, chosen using a random number 
generator with weighted values for the different multi- 
pliers. When the jackpot is more than $150,000,000, the 
weighted values are 24 for 2X, 13 for 3X, 3 for 4X, and 2 
for 5X. When the jackpot does not exceed $150,000,000, 
the weighted values are 24 for 2X, 13 for 3X, 3 for 4X, 2 
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for 5X, and 1 for 10X. All non-jackpot prizes are multi- 
plied by the multiplier chosen, except for the $1,000,000 
prize, which is doubled when the Power Play option is 
in effect regardless of the multiplier chosen. What is the 
probability that a play who has purchased a Powerball 
ticket and Power Play wins 

a) $2,000,000, if the jackpot is more than $150,000,000? 

b) $2,000,000, if the jackpot does not exceed 
$150,000,000? 

c) $1000, if the jackpot does not exceed $150,000,000? 
(The two ways to do this are for the Power Play mul- 
tiplier to be 10X, and to match either exactly four of 
the first five numbers but not the sixth number drawn, 
or exactly three of the first five numbers and the sixth 
number drawn.) 

d) $12, if the jackpot is more than $150,000,000? (The 
two ways to do this are for the Power Play multiplier 
to be 3X and to match the sixth number and either one 
or none of the first five numbers drawn.) 


Two events FE, and EF, are called independent if 
P(E, N E) = p(E, )p(E,). For each of the following pairs 
of events, which are subsets of the set of all possible 
outcomes when a coin is tossed three times, determine 
whether or not they are independent. 


a) E;: tails comes up with the coin is tossed the first 
time; E,: heads comes up when the coin is tossed the 
second time. 

b) E: the first coin comes up tails; E,: two, and not 
three, heads come up in a row. 
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c) E,: the second coin comes up tails; E,: two, and not 
three, heads come up in a row. 

(We will study independence of events in more depth in 

Section 7.2.) 


Explain what is wrong with the statement that in the 
Monty Hall Three-Door Puzzle the probability that the 
prize is behind the first door you select and the probabil- 
ity that the prize is behind the other of the two doors that 
Monty does not open are both 1/2, because there are two 
doors left. 


Suppose that instead of three doors, there are four doors 
in the Monty Hall puzzle. What is the probability that 
you win by not changing once the host, who knows what 
is behind each door, opens a losing door and gives you 
the chance to change doors? What is the probability that 
you win by changing the door you select to one of the two 
remaining doors among the three that you did not select? 


This problem was posed by the Chevalier de Méré and 

was solved by Blaise Pascal and Pierre de Fermat. 

a) Find the probability of rolling at least one six when a 
fair die is rolled four times. 

b) Find the probability that a double six comes up at least 
once when a pair of dice is rolled 24 times. Answer 
the query the Chevalier de Méré made to Pascal ask- 
ing whether this probability was greater than 1/2. 

c) Is it more likely that a six comes up at least once when 
a fair die is rolled four times or that a double six comes 
up at least once when a pair of dice is rolled 24 times? 


7.2.1 Introduction 
Links > 
Laplace did, that is, 


P(E) = a 
Isl’ 


In Section 7.1 we introduced the notion of the probability of an event. (Recall that an event is a 
subset of the possible outcomes of an experiment.) We defined the probability of an event E as 


the number of outcomes in E divided by the total number of outcomes. This definition assumes 
that all outcomes are equally likely. However, many experiments have outcomes that are not 
equally likely. For instance, a coin may be biased so that it comes up heads twice as often as 
tails. Similarly, the likelihood that the input of a linear search is a particular element in a list, 
or is not in the list, depends on how the input is generated. How can we model the likelihood of 
events in such situations? In this section we will show how to define probabilities of outcomes 
to study probabilities of experiments where outcomes may not be equally likely. 

Suppose that a fair coin is flipped four times, and the first time it comes up heads. Given 
this information, what is the probability that heads comes up three times? To answer this and 
similar questions, we will introduce the concept of conditional probability. Does knowing that 
the first flip comes up heads change the probability that heads comes up three times? If not, 
these two events are called independent, a concept studied later in this section. 
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Many questions address a particular numerical value associated with the outcome of 
an experiment. For instance, when we flip a coin 100 times, what is the probability that 
exactly 40 heads appear? How many heads should we expect to appear? In this section we 
will introduce random variables, which are functions that associate numerical values to the 
outcomes of experiments. 


7.2.2 Assigning Probabilities 


Let S be the sample space of an experiment with a finite or countable number of outcomes. We 
assign a probability p(s) to each outcome s. We require that two conditions be met: 


(i) O<p(s)<1foreachs E€ S$ 


and 


Gi) P pal 


ses 


Condition (i) states that the probability of each outcome is a nonnegative real number no greater 
than 1. Condition (ii) states that the sum of the probabilities of all possible outcomes should 
be 1; that is, when we do the experiment, it is a certainty that one of these outcomes occurs. 
(Note that when the sample space is infinite, as p(s) is a convergent infinite series.) This is 
a generalization of Laplace’s definition in which each of n outcomes is assigned a probability 
of 1/n. Indeed, conditions (i) and (ii) are met when Laplace’s definition of probabilities of 
equally likely outcomes is used and S is finite. (See Exercise 4.) 

Note that when there are n possible outcomes, x,, X2, ...,x,, the two conditions to be 
met are 


@) O<p,) <1fori=1,2,...,n 


and 
Gi) J posi 
l 


The function p from the set of all outcomes of the sample space S is called a probability 
distribution. 

To model an experiment, the probability p(s) assigned to an outcome s should equal the limit 
of the number of times s occurs divided by the number of times the experiment is performed, 
because this number grows without bound. (We will assume that all experiments discussed have 
outcomes that are predictable on the average, so that this limit exists. We also assume that the 
outcomes of successive trials of an experiment do not depend on past results.) 


Remark: We will not discuss probabilities of events when the set of outcomes is not finite or 
countable, such as when the outcome of an experiment can be any real number. In such cases, 
integral calculus is usually required for the study of the probabilities of events. 


HISTORICAL NOTE The Chevalier de Méré was a French nobleman, a famous gambler, and a bon vivant. 
He was successful at making bets with odds slightly greater than 1/2 (such as having at least one six come 
up in four tosses of a fair die). His correspondence with Pascal asking about the probability of having at least 
one double six come up when a pair of dice is rolled 24 times led to the development of probability theory. 
According to one account, Pascal wrote to Fermat about the Chevalier saying something like “He’s a good guy 
but, alas, he’s no mathematician.” 
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We can model experiments in which outcomes are either equally likely or not equally likely 
by choosing the appropriate function p(s), as Example 1 illustrates. 


What probabilities should we assign to the outcomes H (heads) and T (tails) when a fair coin 
is flipped? What probabilities should be assigned to these outcomes when the coin is biased so 
that heads comes up twice as often as tails? 


Solution: For a fair coin, the probability that heads comes up when the coin is flipped equals 
the probability that tails comes up, so the outcomes are equally likely. Consequently, we assign 


the probability 1/2 to each of the two possible outcomes, that is, p(H) = p(T) = 1/2. 
For the biased coin we have 


P(A) = 2p(T). 
Because 
P(A) + p(T) = 1, 
it follows that 
2p(T) + p(T) = 3p(T) = 1. 


We conclude that p(T) = 1/3 and p(A) = 2/3. < 


Suppose that S is a set with n elements. The uniform distribution assigns the probability 1/n 
to each element of S. 


We now define the probability of an event as the sum of the probabilities of the outcomes 
in this event. 


The probability of the event E is the sum of the probabilities of the outcomes in E. That is, 


P(E) = 9 p6). 


SCE 


(Note that when £ is an infinite set, De g P(s) is a convergent infinite series.) 


Note that when there are n outcomes in the event E, that is, if E = {a}, a», ..., a,,}, then p(E) = 
È; p(a,). Note also that the uniform distribution assigns the same probability to an event that 
Laplace’s original definition of probability assigns to this event. The experiment of selecting an 
element from a sample space with a uniform distribution is called selecting an element of S at 
random. 


Suppose that a die is biased (or loaded) so that 3 appears twice as often as each other number 


but that the other five outcomes are equally likely. What is the probability that an odd number 
appears when we roll this die? 


Solution: We want to find the probability of the event E = {1, 3,5}. By Exercise 2, we have 


pl) =p) = p(4) = pG6) = p6) = 1/7; p83) = 2/7. 
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It follows that 


P(E) = p(1) + p(3) + p(5) = 1/7 + 2/7 + 1/7 = 4/7. 4 


When possible outcomes are equally likely and there are a finite number of possible out- 
comes, the definition of the probability of an event given in this section (Definition 2) agrees 
with Laplace’s definition (Definition 1 of Section 7.1). To see this, suppose that there are n 
equally likely outcomes; each possible outcome has probability 1/n, because the sum of their 
probabilities is 1. Suppose the event E contains m outcomes. According to Definition 2, 


7 ‘ 


D=) = 
i=1 


Because |E| = m and |S| = n, it follows that 


m JEI 
E = — = —, 
pE) ai 


This is Laplace’s definition of the probability of the event E. 


7.2.3 Probabilities of Complements and Unions of Events 


The formulae for probabilities of combinations of events in Section 7.1 continue to hold 
when we use Definition 2 to define the probability of an event. For example, Theorem 1 of 
Section 7.1 asserts that 


p(E) = 1 — p(E), 


where E is the complementary event of the event E. This equality also holds when Definition 2 
is used. To see this, note that because the sum of the probabilities of the n possible outcomes 


is 1, and each outcome is either in £ or in E, but not in both, we have 


$ ps) =1=pE)+p®). 
seS 


Hence, p(E) = 1 — p(E). 
Under Laplace’s definition, by Theorem 2 in Section 7.1, we have 


P(E: U Ey) = p(E\) + p(E2) — p(E; NE) 


whenever E} and E, are events in a sample space S. This also holds when we define the prob- 
ability of an event as we do in this section. To see this, note that p(E, U E,) is the sum of 
the probabilities of the outcomes in E£; U E. When an outcome x is in one, but not both, 
of E, and £,, p(x) occurs in exactly one of the sums for p(Z,) and p(E,). When an 
outcome x is in both £; and £,, p(x) occurs in the sum for p(£,), in the sum for p(E,), and 
in the sum for p(E, N E,), so it occurs 1 + 1 — 1 = 1 time on the right-hand side. Consequently, 
the left-hand side and right-hand side are equal. 
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Also, note that if the events £, and £E, are disjoint, then p(E, N E,) = 0, which implies that 
P(E U Ey) = p(E\) + p(Ey) — pE, N Ey) = pE) + pE). 


Theorem 1 generalizes this last formula by providing a formula for the probability of the 
union of pairwise disjoint events. 


If E,, E, ... is a sequence of pairwise disjoint events in a sample space S, then 


p (U s) = 2 pÆ). 


(Note that this theorem applies when the sequence E}, E}, ... consists of a finite number or a 
countably infinite number of pairwise disjoint events.) 


We leave the proof of Theorem 1 to the reader (see Exercises 36 and 37). 


7.2.4 Conditional Probability 


Suppose that we flip a coin three times, and all eight possibilities are equally likely. Moreover, 
suppose we know that the event F, that the first flip comes up tails, occurs. Given this infor- 
mation, what is the probability of the event E, that an odd number of tails appears? Because 
the first flip comes up tails, there are only four possible outcomes: TTT, TTH, THT, and THH, 
where H and T represent heads and tails, respectively. An odd number of tails appears only for 
the outcomes TTT and THH. Because the eight outcomes have equal probability, each of the 
four possible outcomes, given that F occurs, should also have an equal probability of 1/4. This 
suggests that we should assign the probability of 2/4 = 1/2 to E, given that F occurs. This 
probability is called the conditional probability of E given F. 

In general, to find the conditional probability of E given F, we use F as the sample space. 
For an outcome from E to occur, this outcome must also belong to E N F. With this motivation, 
we make Definition 3. 


Let E and F be events with p(F) > 0. The conditional probability of E given F, denoted by 
D(E | F), is defined as 


PENAF) 


E| F)= 
P(E | F) DE) 


A bit string of length four is generated at random so that each of the 16 bit strings of length four 
is equally likely. What is the probability that it contains at least two consecutive Os, given that 
its first bit is a 0? (We assume that 0 bits and 1 bits are equally likely.) 


Solution: Let E be the event that a bit string of length four contains at least two consecutive Os, 
and let F be the event that the first bit of a bit string of length four is a 0. The probability that a 
bit string of length four has at least two consecutive Os, given that its first bit is a 0, equals 


PENAF) 


E|F)= 
P(E | F) DE) 


482 7/ Discrete Probability 


EXAMPLE 4 


Links 


> 


Definition 4 


EXAMPLE 5 


Extra 
Examples 


» 


Because E N F = {0000, 0001, 0010, 0011, 0100}, we see that p(E N F) = 5/16. Because there 
are eight bit strings of length four that start with a 0, we have p(F) = 8/16 = 1/2. Consequently, 


_ 5/16 _ 5 
PE sg 8 < 


What is the conditional probability that a family with two children has two boys, given they 
have at least one boy? Assume that each of the possibilities BB, BG, GB, and GG is equally 
likely, where B represents a boy and G represents a girl. (Note that BG represents a family with 
an older boy and a younger girl while GB represents a family with an older girl and a younger 
boy.) 


Solution: Let E be the event that a family with two children has two boys, and let F be 
the event that a family with two children has at least one boy. It follows that E = {BB}, 
F = {BB, BG, GB}, and EN F = {BB}. Because the four possibilities are equally likely, it 
follows that p(F) = 3/4 and p(E N F) = 1/4. We conclude that 


pEnF) 1/41 


PE 5G, A a 4 


7.2.5 Independence 


Suppose a coin is flipped three times, as described in the introduction to our discussion of 
conditional probability. Does knowing that the first flip comes up tails (event F) alter the prob- 
ability that tails comes up an odd number of times (event E)? In other words, is it the case 
that p(E | F) = p(E)? This equality is valid for the events E and F, because p(E | F) = 1/2 and 
p(E) = 1/2. Because this equality holds, we say that E and F are independent events. When 
two events are independent, the occurrence of one of the events gives no information about the 
probability that the other event occurs. 

Because p(E | F) = p(E N F)/p(F), asking whether p(E | F) = p(E) is the same as asking 
whether p(E N F) = p(E)p(F). This leads to Definition 4. 


The events E and F are independent if and only if p(E N F) = p(E)p(F). 


Suppose ŒE is the event that a randomly generated bit string of length four begins with a 1 
and F is the event that this bit string contains an even number of 1s. Are E and F independent, 
if the 16 bit strings of length four are equally likely? 


Solution: There are eight bit strings of length four that begin with a one: 1000, 1001, 1010, 
1011, 1100, 1101, 1110, and 1111. There are also eight bit strings of length four that contain 
an even number of ones: 0000, 0011, 0101, 0110, 1001, 1010, 1100, 1111. Because there are 
16 bit strings of length four, it follows that 

pÆ) = p(F) = 8/16 = 1/2. 
Because EN F = {1111, 1100, 1010, 1001}, we see that 


P(E NF) = 4/16 = 1/4. 
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Because 
PEN F) = 1/4 = (1/2)(1/2) = p(E)p(F), 


we conclude that E and F are independent. < 


Probability has many applications to genetics, as Examples 6 and 7 illustrate. 


Assume, as in Example 4, that each of the four ways a family can have two children is equally 
likely. Are the events E, that a family with two children has two boys, and F, that a family with 
two children has at least one boy, independent? 


Solution: Because E = {BB}, we have p(E) = 1/4. In Example 4 we showed that p(F) = 3/4 
and that p(E Nn F) = 1/4. But p(E)p(F) = L : 2 = =. Therefore, p(E N F) # p(E)p(F), so the 
events E and F are not independent. < 


Are the events E, that a family with three children has children of both sexes, and F, that this 
family has at most one boy, independent? Assume that the eight ways a family can have three 
children are equally likely. 


Solution: By assumption, each of the eight ways a family can have three children, 
BBB, BBG, BGB, BGG, GBB, GBG, GGB, and GGG, has a probability of 1/8. Because 
E = {BBG, BGB, BGG, GBB, GBG, GGB}, F= {BGG, GBG, GGB, GGG}, and 
ENF = {BGG, GBG, GGB}, it follows that p(E) = 6/8 = 3/4, p(F) = 4/8 = 1/2, and 
p(E N F) = 3/8. Because 


p(E)p(F) = 


it follows that p(E N F) = p(E)p(F), so E and F are independent. (This conclusion may seem 
surprising. Indeed, if we change the number of children, the conclusion may no longer hold. 
See Exercise 27.) < 


PAIRWISE AND MUTUAL INDEPENDENCE We can also define the independence of more 
than two events. However, there are two different types of independence, given in Definition 5. 


The events E, Ep... E, are pairwise independent if and only if p(E;N E, = 
p(E;)p(E;) for all pairs of integers i and j with 1<i<j<n. These events are 
mutually independent if pE; NE, N+ NE; ) = pE pE) pE) whenever 
lod = 1,2,...,m, are integers with 1 <i, <i, <--- <i, <nandm> 2. 


We 


From Definition 5, we see that every set of n mutually independent events is also pairwise 
independent. However, n pairwise independent events are not necessarily mutually independent, 
as we see in Exercise 25 in the Supplementary Exercises. Many theorems about n events include 
the hypothesis that these events are mutually independent, and not just pairwise independent. 
We will introduce several such theorems later in this chapter. 
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7.2.6 Bernoulli Trials and the Binomial Distribution 


Suppose that an experiment can have only two possible outcomes. For instance, when a bit is 
generated at random, the possible outcomes are 0 and 1. When a coin is flipped, the possible 
outcomes are heads and tails. Each performance of an experiment with two possible outcomes is 
called a Bernoulli trial, after James Bernoulli, who made important contributions to probability 
theory. In general, a possible outcome of a Bernoulli trial is called a success or a failure. If p 
is the probability of a success and q is the probability of a failure, it follows that p + q = 1. 

Many problems can be solved by determining the probability of k successes when an exper- 
iment consists of n mutually independent Bernoulli trials. (Bernoulli trials are mutually inde- 
pendent if the conditional probability of success on any given trial is p, given any information 
whatsoever about the outcomes of the other trials.) Consider Example 8. 


A coin is biased so that the probability of heads is 2/3. What is the probability that exactly four 
heads come up when the coin is flipped seven times, assuming that the flips are independent? 


Solution: There are 2” = 128 possible outcomes when a coin is flipped seven times. The number 
of ways four of the seven flips can be heads is C(7, 4). Because the seven flips are independent, 
the probability of each of these outcomes (four heads and three tails) is (2/ 3d / 3)°. Conse- 
quently, the probability that exactly four heads appear is 


35-16 _ 560 
37. 2187 4 


C(7, 4)(2/3)*(1/3)° = 


Following the same reasoning as was used in Example 8, we can find the probability of k 
successes in n independent Bernoulli trials. 


The probability of exactly k successes in n independent Bernoulli trials, with probability of 
success p and probability of failure g = 1 — p, is 


Cla, kip a, 


Proof: When n Bernoulli trials are carried out, the outcome is an n-tuple (t,t, ..., tn), 
where t; = S (for success) or t; = F (for failure) for i = 1, 2, ..., n. Because the n trials are inde- 
pendent, the probability of each outcome of n trials consisting of k successes and n — k failures 
(in any order) is p<q"~*. Because there are C(n, k) n-tuples of S’s and F’s that contain exactly k 
S’s, the probability of exactly k successes is 


C(n, k)p*q"* : < 


JAMES BERNOULLI (1654-1705) James Bernoulli (also known as Jacob J), was born in Basel, Switzerland. 
He is one of the eight prominent mathematicians in the Bernoulli family (see Section 10.1 for the Bernoulli fam- 
ily tree of mathematicians). Following his father’s wish, James studied theology and entered the ministry. But 
contrary to the desires of his parents, he also studied mathematics and astronomy. He traveled throughout Eu- 
rope from 1676 to 1682, learning about the latest discoveries in mathematics and the sciences. Upon returning 
to Basel in 1682, he founded a school for mathematics and the sciences. He was appointed professor of mathe- 
matics at the University of Basel in 1687, remaining in this position for the rest of his life. 

James Bernoulli is best known for the work Ars Conjectandi, published eight years after his death. In 


©INTERFOTO/Alamy Stock this work, he described the known results in probability theory and in enumeration, often providing alternative 


Photo 


proofs of known results. This work also includes the application of probability theory to games of chance and his 


introduction of the theorem known as the law of large numbers. This law states that if e > 0, as n becomes arbitrarily large the 
probability approaches 1 that the fraction of times an event E occurs during n trials is within e of p(E). 
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We denote by b(k;n, p) the probability of k successes in n independent Bernoulli trials with 
probability of success p and probability of failure g = 1 — p. Considered as a function of k, we 
call this function the binomial distribution. Theorem 2 tells us that b(k; n, p) = C(n, kpt". 


Suppose that the probability that a 0 bit is generated is 0.9, that the probability that a 1 bit is 
generated is 0.1, and that bits are generated independently. What is the probability that exactly 
eight 0 bits are generated when 10 bits are generated? 


Solution: By Theorem 2, the probability that exactly eight 0 bits are generated is 


b(8; 10, 0.9) = C(10, 8)(0.9)8(0.1)? = 0.1937102445. < 


Note that the sum of the probabilities that there are k successes when n independent Bernoulli 
trials are carried out, for k = 0, 1, 2, ...,, equals 


È, Ce, Wp'a* = p +q =1, 
k=0 


as should be the case. The first equality in this string of equalities is a consequence of the 
binomial theorem (see Section 6.4). The second equality follows because q = | — p. 


7.2.7 Random Variables 


Many problems are concerned with a numerical value associated with the outcome of an experi- 
ment. For instance, we may be interested in the total number of one bits in a randomly generated 
string of 10 bits; or in the number of times tails come up when a coin is flipped 20 times. To 
study problems of this type we introduce the concept of a random variable. 


A random variable is a function from the sample space of an experiment to the set of real 
numbers. That is, a random variable assigns a real number to each possible outcome. 


Remark: Note that a random variable is a function. It is not a variable, and it is not random! 
The name random variable (the translation of variabile casuale) was introduced by the Ital- 
ian mathematician F. P. Cantelli in 1916. In the late 1940s, the mathematicians, W. Feller and 
J. L. Doob flipped a coin to see whether both would use “random variable” or the more fitting 
term “chance variable.” Feller won; unfortunately “random varible” was used in both books and 
ever since. 


Suppose that a coin is flipped three times. Let X(t) be the random variable that equals the number 
of heads that appear when f is the outcome. Then X(f) takes on the following values: 


X(HHH) = 3, 
X(HHT) = X(HTH) = X(THH) = 2, 
X(TTH) = X(THT) = X(HTT) = 1, 
X(TIT) = 0. < 
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Links > 


The distribution of a random variable X on a sample space S is the set of pairs (r, p(X = r)) 
for all r € X(S), where p(X = r) is the probability that X takes the value r. (The set of pairs 
in this distribution is determined by the probabilities p(X = r) for r € X(S).) 


Each of the eight possible outcomes when a fair coin is flipped three times has probability 
1/8. So, the distribution of the random variable X(t) in Example 10 is determined by the 
probabilities P(X = 3) = 1/8, P(X = 2) = 3/8, P(X = 1) = 3/8, and P(X = 0) = 1/8. Conse- 
quently, the distribution of X(t) in Example 10 is the set of pairs (3, 1/8), (2, 3/8), (1, 3/8), 
and (0, 1/8). < 


Let X be the sum of the numbers that appear when a pair of dice is rolled. What are the values 
of this random variable for the 36 possible outcomes (i, j), where i and j are the numbers that 
appear on the first die and the second die, respectively, when these two dice are rolled? 


Solution: The random variable X takes on the following values: 


X, 1)) = 2, 

X, 2)) = X(@, 1)) = 3, 

X((1, 3)) = X(C, 2)) = X(3, D) = 4, 

X, 4) = X(Q, 3) = X(G, D) = X(4, D) =5, 

X, 5)) = X(C, 4)) = X(G, 3) = X(4, 2)) = XS, D) = 6, 

X((1, 6)) = X(2, 5)) = X((3, 4) = X(4, 3)) = XS, 2)) = X(6, D) =7, 
X((2, 6)) = X((3, 5)) = X(4, 4)) = X(S, 3)) = X((6, 2)) = 8, 

X(G, 6)) = X((4, 5) = X(S, 4)) = X(6, 3)) = 9, 

X((4, 6)) = X((5, 5) = X((6, 4)) = 10, 

X((5, 6)) = X((6, 5)) = 11, 

X((6, 6)) = 12. < 


We will continue our study of random variables in Section 7.4, where we will show how 
they can be used in a variety of applications. 


7.2.8 The Birthday Problem 


A famous puzzle asks for the smallest number of people needed in a room so that it is more likely 
than not that at least two of them have the same day of the year as their birthday. Most people 
find the answer, which we determine in Example 13, to be surprisingly small. After we solve 
this famous problem, we will show how similar reasoning can be adapted to solve a question 
about hashing functions. 


The Birthday Problem What is the minimum number of people who need to be in a room 
so that the probability that at least two of them have the same birthday is greater than 1/2? 


Solution: First, we state some assumptions. We assume that the birthdays of the people in the 
room are independent. Furthermore, we assume that each birthday is equally likely and that 
there are 366 days in the year. (In reality, more people are born on some days of the year than 


EXAMPLE 14 
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others, such as days nine months after some holidays including New Year’s Eve, and only leap 
years have 366 days.) 

To find the probability that at least two of n people in a room have the same birthday, 
we first calculate the probability p, that these people all have different birthdays. Then, the 
probability that at least two people have the same birthday is 1- p„. To compute p,,, we consider 
the birthdays of the n people in some fixed order. Imagine them entering the room one at a time; 
we will compute the probability that each successive person entering the room has a birthday 
different from those of the people already in the room. 

The birthday of the first person certainly does not match the birthday of someone already in 
the room. The probability that the birthday of the second person is different from that of the first 
person is 365/366 because the second person has a different birthday when he or she was born 
on one of the 365 days of the year other than the day the first person was born. (The assumption 
that it is equally likely for someone to be born on any of the 366 days of the year enters into this 
and subsequent steps.) 

The probability that the third person has a birthday different from both the birthdays of 
the first and second people given that these two people have different birthdays is 364/366. In 
general, the probability that the jth person, with 2 < j < 366, has a birthday different from the 
birthdays of the j — 1 people already in the room given that these j — 1 people have different 
birthdays is 


366-(j-1) _ 367-j 
366 366 ` 


Because we have assumed that the birthdays of the people in the room are independent, we 
can conclude that the probability that the n people in the room have different birthdays is 


_ 365 364 363 367-n 


Pn = 366 366 366. 366 


It follows that the probability that among n people there are at least two people with the same 
birthday is 


365 364 363 367-n 


1- Shee ee 


Pn" ~ 366 366 366. 366 


To determine the minimum number of people in the room so that the probability that at 
least two of them have the same birthday is greater than 1/2, we use the formula we have found 
for 1 — p, to compute it for increasing values of n until it becomes greater than 1/2. (There are 
more sophisticated approaches using calculus that can eliminate this computation, but we will 
not use them here.) After considerable computation we find that for n = 22, 1 — p, ~ 0.475, 
while for n = 23, 1—p,, ~ 0.506. Consequently, the minimum number of people needed so that 
the probability that at least two people have the same birthday is greater than 1/2 is 23. < 


The solution to the birthday problem leads to the solution of the question in Example 14 
about hashing functions. 


Probability of a Collision in Hashing Functions Recall from Section 4.5 that a hashing 
function h(k) is a mapping of the keys (of the records that are to be stored in a database) to 
storage locations. Hashing functions map a large universe of keys (such as the approximately 
300 million Social Security numbers in the United States) to a much smaller set of storage 
locations. A good hashing function yields few collisions, which are mappings of two different 
keys to the same memory location, when relatively few of the records are in play in a given 
application. What is the probability that no two keys are mapped to the same location by a 
hashing function, or, in other words, that there are no collisions? 
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Solution: To calculate this probability, we assume that the probability that a randomly selected 
key is mapped to a location is 1/m, where m is the number of available locations, that is, the 
hashing function distributes keys uniformly. (In practice, hashing functions may not satisfy this 
assumption. However, for a good hashing function, this assumption should be close to correct.) 
Furthermore, we assume that the keys of the records selected have an equal probability to be 
any of the elements of the key universe and that these keys are independently selected. 

Suppose that the keys are k,, kz, ...,k,. When we add the second record, the probability 
that it is mapped to a location different from the location of the first record, that h(k,) # h(k,), 
is (m — 1)/m because there are m — 1 free locations after the first record has been placed. The 
probability that the third record is mapped to a free location after the first and second records 
have been placed without a collision is (m — 2)/m. In general, the probability that the jth record 
is mapped to a free location after the first j — 1 records have been mapped to locations h(k,), 
h(ky), ..., h(kj_-1) without collisions is (m — (j — 1))/m because j — 1 of the m locations are 
taken. 

Because the keys are independent, the probability that all n keys are mapped to different 
locations is 


_m-1 m-2 m-n+1 
Pn = à s ovs s . 
m m m 
It follows that the probability that there is at least one collision, that is, at least two keys are 
mapped to the same location, is 


Techniques from calculus can be used to find the smallest value of n given a value of m such 
that the probability of a collision is greater than a particular threshold. It can be shown that the 
smallest integer n such that the probability of a collision is greater than 1/2 is approximately 
n=1.177 „ym. For example, when m = 1,000,000, the smallest integer n such that the proba- 
bility of a collision is greater than 1/2 is 1178. < 


7.2.9 Monte Carlo Algorithms 


The algorithms discussed so far in this book are all deterministic. That is, each algorithm always 
proceeds in the same way whenever given the same input. However, there are many situations 
where we would like an algorithm to make a random choice at one or more steps. Such a situ- 
ation arises when a deterministic algorithm would have to go through a huge number, or even 
an unknown number, of possible cases. Algorithms that make random choices at one or more 
steps are called probabilistic algorithms. We will discuss a particular class of probabilistic al- 
gorithms in this section, namely, Monte Carlo algorithms, for decision problems. Monte Carlo 
algorithms always produce answers to problems, but a small probability remains that these an- 
swers may be incorrect. However, the probability that the answer is incorrect decreases rapidly 
when the algorithm carries out sufficient computation. Decision problems have either “true” or 
“false” as their answer. The designation “Monte Carlo” is a reference to the gambling casino 
in Monaco; the use of randomness and the repetitive processes in these algorithms make them 
similar to some gambling games. This name was introduced by the inventors of Monte Carlo 
methods, including Stan Ulam, Enrico Fermi, and John von Neumann. 

A Monte Carlo algorithm for a decision problem uses a sequence of tests. The probability 
that the algorithm answers the decision problem correctly increases as more tests are carried 
out. At each step of the algorithm, possible responses are “true,” which means that the answer is 
“true” and no additional iterations are needed, or “unknown,” which means that the answer could 
be either “true” or “false.” After running all the iterations in such an algorithm, the final answer 
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produced is “true” if at least one iteration yields the answer “true,” and the answer is “false” if 
every iteration yields the answer “unknown.” If the correct answer is “false,” then the algorithm 
answers “false,” because every iteration will yield “unknown.” However, if the correct answer is 
“true,” then the algorithm could answer either “true” or “false,” because it may be possible that 
each iteration produced the response “unknown” even though the correct response was “true.” 
We will show that this possibility becomes extremely unlikely as the number of tests increases. 
Suppose that p is the probability that the response of a test is “true,” given that the answer 
is “true.” It follows that 1—p is the probability that the response is “unknown,” given that the 
answer is “true.” Because the algorithm answers “false” when all n iterations yield the answer 
“unknown” and the iterations perform independent tests, the probability of error is (1—p)”. 
When p # 0, this probability approaches 0 as the number of tests increases. Consequently, the 
probability that the algorithm answers “true” when the answer is “true” approaches 1. 


Quality Control (This example is adapted from [AhU195].) Suppose that a manufacturer or- 
ders processor chips in batches of size n, where n is a positive integer. The chip maker has 
tested only some of these batches to make sure that all the chips in the batch are good (re- 
placing any bad chips found during testing with good ones). In previously untested batches, 
the probability that a particular chip is bad has been observed to be 0.1 when random testing 
is done. The PC manufacturer wants to decide whether all the chips in a batch are good. To 
do this, the PC manufacturer can test each chip in a batch to see whether it is good. However, 
this requires n tests. Assuming that each test can be carried out in constant time, these tests 
require O(n) seconds. Can the PC manufacturer determine whether a batch of chips has been 
tested by the chip maker using less time? 


Solution: We can use a Monte Carlo algorithm to determine whether a batch of chips has been 
tested by the chip maker as long as we are willing to accept some probability of error. The 
algorithm is set up to answer the question: “Has this batch of chips not been tested by the 
chip maker?” It proceeds by successively selecting chips at random from the batch and testing 
them one by one. When a bad chip is encountered, the algorithm answers “true” and stops. If 
a tested chip is good, the algorithm answers “unknown” and goes on to the next chip. After 
the algorithm has tested a specified number of chips, say k chips, without getting an answer of 
“true,” the algorithm terminates with the answer “false”; that is, the algorithm concludes that 
the batch is good, that is, that the chip maker has tested all the chips in the batch. 

The only way for this algorithm to answer incorrectly is for it to conclude that an untested 
batch of chips has been tested by the chip maker. The probability that a chip is good, but that 
it came from an untested batch, is 1 — 0.1 = 0.9. Because the events of testing different chips 
from a batch are independent, the probability that all k steps of the algorithm produce the answer 
“unknown,” given that the batch of chips is untested, is 0.9%. 

By taking k large enough, we can make this probability as small as we like. For example, 
by testing 66 chips, the probability that the algorithm decides a batch has been tested by the 
chip maker is 0.96€, which is less than 0.001. That is, the probability is less than 1 in 1000 
that the algorithm has answered incorrectly. Note that this probability is independent of n, the 
number of chips in a batch. That is, the Monte Carlo algorithm uses a constant number, or O(1), 
tests and requires O(1) seconds, no matter how many chips are in a batch. As long as the PC 
manufacturer can live with an error rate of less than 1 in 1000, the Monte Carlo algorithm will 
save the PC manufacturer a lot of testing. If a smaller error rate is needed, the PC manufacturer 
can test more chips in each batch; the reader can verify that 132 tests lower the error rate to less 
than 1 in 1,000,000. < 


Probabilistic Primality Testing In Chapter 4 we remarked that a composite integer, that is, an 
integer greater than one that is not prime, passes Miller’s test (see the preamble to Exercise 44 
in Section 4.4) for fewer than n/4 bases b with 1 < b <n. This observation is the basis for 
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THEOREM 4 


a Monte Carlo algorithm to determine whether an integer greater than one is prime. Because 
large primes play an essential role in public-key cryptography (see Section 4.6), being able to 
generate large primes quickly has become extremely important. 

The goal of the algorithm is to decide the question “Is n composite?” Given an integer 
n greater than one, we select an integer b at random with 1 < b <n and determine whether 
n passes Miller’s test to the base b. If n fails the test, the answer is “true” because n must be 
composite, and the algorithm ends. Otherwise, we perform the test k times, where k is a positive 
integer. Each time we select a random integer b and determine whether n passes Miller’s test to 
the base b. If the answer is “unknown” at each step, the algorithm answers “false,” that is, it says 
that n is not composite, so that it is prime. The only possibility for the algorithm to return an 
incorrect answer occurs when n is composite, and the answer “unknown” is the output at each 
of the k iterations. The probability that a composite integer n passes Miller’s test for a randomly 
selected base b is less than 1/4. Because the integer b with 1 < b < n is selected at random at 
each iteration and these iterations are independent, the probability that n is composite but the 
algorithm responds that n is prime is less than (1/4)*. By taking k to be sufficiently large, we 
can make this probability extremely small. For example, with 10 iterations, the probability that 
the algorithm decides that n is prime when it really is composite is less than 1 in 1,000,000. 
With 30 iterations, this probability drops to less than 1 in 10!8, an extremely unlikely event. 

To generate large primes, say with 200 digits, we randomly choose an integer n with 200 
digits and run this algorithm, with 30 iterations. If the algorithm decides that n is prime, we 
can use it as one of the two primes used in an encryption key for the RSA cryptosystem. If n is 
actually composite and is used as part of the key, the procedures used to decrypt messages will 
not produce the original encrypted message. The key is then discarded and two new possible 
primes are used. < 


7.2.10 The Probabilistic Method 


We discussed existence proofs in Chapter 1 and illustrated the difference between constructive 
existence proofs and nonconstructive existence proofs. The probabilistic method, introduced by 
Paul Erdős and Alfréd Rényi, is a powerful technique that can be used to create nonconstructive 
existence proofs. To use the probabilistic method to prove results about a set S, such as the 
existence of an element in S with a specified property, we assign probabilities to the elements 
of S. We then use methods from probability theory to prove results about the elements of S. In 
particular, we can show that an element with a specified property exists by showing that the 
probability an element x € S has this property is positive. The probabilistic method is based on 
the equivalent statement in Theorem 3. 


THE PROBABILISTIC METHOD If the probability that an element chosen at random 
from a S does not have a particular property is less than 1, there exists an element in S with 
this property. 


An existence proof based on the probabilistic method is nonconstructive because it does not 
find a particular element with the desired property. 

We illustrate the power of the probabilistic method by finding a lower bound for the Ramsey 
number R(k, k). Recall from Section 6.2 that R(k, k) equals the minimum number of people at 
a party needed to ensure that there are at least k mutual friends or k mutual enemies (assuming 
that any two people are friends or enemies). 


If k is an integer with k > 2, then R(k, k) > 2%. 


Exercises 


1. What probability should be assigned to the outcome of 
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Proof: We note that the theorem holds for k = 2 and k = 3 because R(2, 2) = 2 and R(3, 3) = 6 
as was shown in Section 6.2. Now suppose that k > 4. We will use the probabilistic method to 
show that if there are fewer than 2‘/? people at a party, it is possible that no k of them are mutual 
friends or mutual enemies. This will show that R(k, k) is at least 2*/?. 

To use the probabilistic method, we assume that it is equally likely for two people to be 
friends or enemies. (Note that this assumption does not have to be realistic.) Suppose there 
are n people at the party. It follows that there are (X) different sets of k people at this 
party, which we list as S4, Sz, Or Let E; be the event that all k people in S; are ei- 
ther mutual friends or mutual enemies. The probability that there are either k mutual friends 
or k mutual enemies among the n people equals p(U;“, Ep. 

According to our assumption it is equally likely for two people to be friends or enemies. 
The probability that two people are friends equals the probability that they are enemies; both 
probabilities equal 1/2. Furthermore, there are () = k(k—1)/2 pairs of people in S; because 
there are k people in S;. Hence, the probability that all k people in S, are mutual friends and the 
probability that all k people in S; are mutual enemies both equal (1/2)**-)/?. It follows that 
p(E,) = 201/2)K@-D/?, 

The anaes that there are either k mutual friends or k mutual enemies in the group of n 


people equals p(U Q E;). Using Boole’s inequality (Exercise 15), it follows that 


G Q) 


k(k-1)/2 
p Us < Xoe (1) (3) 1) : 


By Exercise 21 in Section 6.4, we have “ ) < n*/2%!. Hence, 


k(k-1)/2 k k(k-1)/2 
Po sa 
k 2 2k-1 \2 


Now if n < 2*/2, we have 


nk me k(k-1)/2 ` wa peoe _ Ht) 2] 
2 Qk-1 2 T? 


where the last step follows because k > 4. 


We can now conclude that pU® a) E;) < 1 when k > 4. Hence, the probability of the com- 
plementary event, that there is no set of either k mutual friends or mutual enemies at the 
party, is greater than 0. It follows that if n < 2*/?, there is at least one set such that no subset 
of k people are mutual friends or mutual enemies. < 


as rolling each of the other four numbers on the die and 


heads when a biased coin is tossed, if heads is three times 
as likely to come up as tails? What probability should be 
assigned to the outcome of tails? 

. Find the probability of each outcome when a loaded die 
is rolled, if a 3 is twice as likely to appear as each of the 
other five numbers on the die. 

. Find the probability of each outcome when a biased die 
is rolled, if rolling a 2 or rolling a 4 is three times as likely 


it is equally likely to roll a 2 or a 4. 


. Show that conditions (i) and (ii) are met under Laplace’s 


definition of probability, when outcomes are equally 
likely. 


. A pair of dice is loaded. The probability that a 4 appears 


on the first die is 2/7, and the probability that a 3 appears 
on the second die is 2/7. Other outcomes for each die 
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10. 


11. 


12. 


13. 


appear with probability 1/7. What is the probability of 7 
appearing as the sum of the numbers when the two dice 
are rolled? 


. What is the probability of these events when we randomly 


select a permutation of {1, 2,3}? 
a) 1 precedes 3. 

b) 3 precedes 1. 

c) 3 precedes 1 and 3 precedes 2. 


. What is the probability of these events when we randomly 


select a permutation of {1, 2, 3, 4}? 

a) 1 precedes 4. 

b) 4 precedes 1. 

c) 4 precedes 1 and 4 precedes 2. 

d) 4 precedes 1, 4 precedes 2, and 4 precedes 3. 
e) 4 precedes 3 and 2 precedes 1. 


. What is the probability of these events when we randomly 


select a permutation of {1, 2, ..., n} where n > 4? 
a) | precedes 2. 

b) 2 precedes 1. 

c) 1 immediately precedes 2. 

d) n precedes 1 and n — 1 precedes 2. 

e) n precedes 1 and n precedes 2. 


. What is the probability of these events when we randomly 


select a permutation of the 26 lowercase letters of the En- 

glish alphabet? 

a) The permutation consists of the letters in reverse al- 
phabetic order. 

b) zis the first letter of the permutation. 

c) z precedes a in the permutation. 

d) aimmediately precedes z in the permutation. 

e) a immediately precedes m, which immediately pre- 
cedes z in the permutation. 

f) m, n, and o are in their original places in the permu- 
tation. 

What is the probability of these events when we randomly 

select a permutation of the 26 lowercase letters of the En- 

glish alphabet? 

a) The first 13 letters of the permutation are in alphabet- 
ical order. 

b) ais the first letter of the permutation and z is the last 
letter. 

c) aand z are next to each other in the permutation. 

d) a and b are not next to each other in the permutation. 

e) a and z are separated by at least 23 letters in the per- 
mutation. 

f) z precedes both a and b in the permutation. 

Suppose that E and F are events such that p(E) = 

0.7 and p(F)=0.5. Show that p(EUF)>0.7 and 

P(E N F) = 0.2. 

Suppose that E and F are events such that p(E) = 

0.8 and p(F) =0.6. Show that p(EUF)>0.8 and 

P(EN F) = 0.4. 

Show that if E and F are events, then p(E N F) > p(E) + 

p(F) — 1. This is known as Bonferroni’s inequality. 


14. 


15, 


16. 


17. 


Use mathematical induction to prove the following gen- 
eralization of Bonferroni’s inequality: 


pE NE, N+ N E,) 
> pEi) + p(Ey) ++ + p(E,) - (1-1), 


where E}, E, ..., E, are n events. 
Show that if £., E,,..., E, are events from a finite sample 
space, then 


P(E, U E U + U E) 
< P(E) + p(E,) + + + pE). 
This is known as Boole’s inequality. 
Show that if E and F are independent events, then E 
and F are also independent events. 
If E and F are independent events, prove or disprove 
that E and F are necessarily independent events. 


In Exercises 18, 20, and 21 assume that the year has 366 days 
and all birthdays are equally likely. In Exercise 19 assume it 
is equally likely that a person is born in any given month of 
the year. 


18. 


19. 


20. 


21. 


*22. 


23. 


24. 


a) What is the probability that two people chosen at ran- 
dom were born on the same day of the week? 

b) What is the probability that in a group of n people 
chosen at random, there are at least two born on the 
same day of the week? 

c) How many people chosen at random are needed to 
make the probability greater than 1/2 that there are at 
least two people born on the same day of the week? 

a) What is the probability that two people chosen at ran- 
dom were born during the same month of the year? 

b) What is the probability that in a group of n people 
chosen at random, there are at least two born in the 
same month of the year? 

c) How many people chosen at random are needed to 
make the probability greater than 1/2 that there are at 
least two people born in the same month of the year? 

Find the smallest number of people you need to choose 

at random so that the probability that at least one of them 

has a birthday today exceeds 1/2. 

Find the smallest number of people you need to choose 

at random so that the probability that at least two of them 

were both born on April 1 exceeds 1/2. 

February 29 occurs only in leap years. Years divisible 

by 4, but not by 100, are always leap years. Years divis- 

ible by 100, but not by 400, are not leap years, but years 
divisible by 400 are leap years. 

a) What probability distribution for birthdays should be 
used to reflect how often February 29 occurs? 

b) Using the probability distribution from part (a), what 
is the probability that in a group of n people at least 
two have the same birthday? 

What is the conditional probability that exactly four 

heads appear when a fair coin is flipped five times, given 

that the first flip came up heads? 

What is the conditional probability that exactly four 

heads appear when a fair coin is flipped five times, given 

that the first flip came up tails? 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


What is the conditional probability that a randomly gen- 

erated bit string of length four contains at least two con- 

secutive Os, given that the first bit is a 1? (Assume the 

probabilities of a 0 and a 1 are the same.) 

Let E be the event that a randomly generated bit string 

of length three contains an odd number of 1s, and let F 

be the event that the string starts with 1. Are E and F 

independent? 

Let E and F be the events that a family of n children has 

children of both sexes and has at most one boy, respec- 

tively. Are E and F independent if 

a) n=2? b) n= 4? c) n=5? 

Assume that the probability a child is a boy is 0.51 

and that the sexes of children born into a family are 

independent. What is the probability that a family of five 

children has 

a) exactly three boys? 

b) at least one boy? 

c) at least one girl? 

d) all children of the same sex? 

A group of six people play the game of “odd person out” 

to determine who will buy refreshments. Each person 

flips a fair coin. If there is a person whose outcome is 

not the same as that of any other member of the group, 

this person has to buy the refreshments. What is the prob- 

ability that there is an odd person out after the coins are 

flipped once? 

Find the probability that a randomly generated bit string 

of length 10 does not contain a 0 if bits are independent 

and if 

a) a0 bit anda 1 bit are equally likely. 

b) the probability that a bit is a 1 is 0.6. 

c) the probability that the ith bit is a 1 is 1/2! for i= 
1,25 35.6 2<.5.10: 

Find the probability that a family with five children does 

not have a boy, if the sexes of children are independent 

and if 

a) a boy and a girl are equally likely. 

b) the probability of a boy is 0.51. 

c) the probability that the ith child is a boy is 
0.51 — (i/100). 

Find the probability that a randomly generated bit string 

of length 10 begins with a | or ends with a 00 for the 

same conditions as in parts (a), (b), and (c) of Exercise 

30, if bits are generated independently. 

Find the probability that the first child of a family with 

five children is a boy or that the last two children of the 

family are girls, for the same conditions as in parts (a), 

(b), and (c) of Exercise 31. 

Find each of the following probabilities when n indepen- 

dent Bernoulli trials are carried out with probability of 

success p. 

a) the probability of no successes 

b) the probability of at least one success 

c) the probability of at most one success 

d) the probability of at least two successes 


35. 


36. 


* 37, 


38. 


#39, 
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Find each of the following probabilities when n indepen- 
dent Bernoulli trials are carried out with probability of 
success p. 


a) the probability of no failures 
b) the probability of at least one failure 
c) the probability of at most one failure 


d) the probability of at least two failures 
Use mathematical induction to prove that if £}, E>, ..., E 


$ n 


is a sequence of n pairwise disjoint events in a sample 
space S, where n is a positive integer, then pU E) = 
(Requires calculus) Show that if E,, E, ... is an infinite 
sequence of pairwise disjoint events in a sample space 
S, then p(U=, E) = D2, pŒ. [Hint: Use Exercise 36 
and take limits.] 


A pair of dice is rolled in a remote location and when you 

ask an honest observer whether at least one die came up 

six, this honest observer answers in the affirmative. 

a) What is the probability that the sum of the numbers 
that came up on the two dice is seven, given the infor- 
mation provided by the honest observer? 


b) Suppose that the honest observer tells us that at least 
one die came up five. What is the probability the sum 
of the numbers that came up on the dice is seven, 
given this information? 

This exercise employs the probabilistic method to prove a 

result about round-robin tournaments. In a round-robin 

tournament with m players, every two players play one 
game in which one player wins and the other loses. 

We want to find conditions on positive integers m 
and k with k < m such that it is possible for the outcomes 
of the tournament to have the property that for every set 
of k players, there is a player who beats every member 
in this set. So that we can use probabilistic reasoning to 
draw conclusions about round-robin tournaments, we as- 
sume that when two players compete it is equally likely 
that either player wins the game and we assume that the 
outcomes of different games are independent. Let E be 
the event that for every set S with k players, where k is 
a positive integer less than m, there is a player who has 
beaten all k players in S. 

D 


a) Show that p(E) < p aaa PF), where F is the event 
that there is no player who beats all k players from 
the jth set in a list of the (5) sets of k players. 


b) Show that the probability of F; is (1 27k, 


c) Conclude from parts (a) and (b) that pE) < 
(are! —2-*y"-k and, therefore, that there must 


be a tournament with the described property if 
es es sae 


d) Use part (c) to find values of m such that there is a 
tournament with m players such that for every set S 
of two players, there is a player who has beaten both 
players in S. Repeat for sets of three players. 
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*40. Devise a Monte Carlo algorithm that determines whether 


a permutation of the integers 1 through n has already 
been sorted (that is, it is in increasing order), or in- 
stead, is a random permutation. A step of the algo- 


“unknown” in each step. Show that as the number of steps 
increases, the probability that the algorithm produces an 
incorrect answer is extremely small. [Hint: For each step, 
test whether certain elements are in the correct order. 


rithm should answer “true” if it determines the list is not Make sure these tests are independent. ] 


sorted and “unknown” otherwise. After k steps, the algo- 41. Use pseudocode to write out the probabilistic primality 
rithm decides that the integers are sorted if the answer is test described in Example 16. 


Bayes’ Theorem 


7.3.1 Introduction 


There are many times when we want to assess the probability that a particular event occurs on 
the basis of partial evidence. For example, suppose we know the percentage of people who have 
a particular disease for which there is a very accurate diagnostic test. People who test positive for 
this disease would like to know the likelihood that they actually have the disease. In this section 
we introduce a result that can be used to determine this probability, namely, the probability that 
a person has the disease given that this person tests positive for it. To use this result, we will 
need to know the percentage of people who do not have the disease but test positive for it and 
the percentage of people who have the disease but test negative for it. 

Similarly, suppose we know the percentage of incoming e-mail messages that are spam. 
We will see that we can determine the likelihood that an incoming e-mail message is spam 
using the occurrence of words in the message. To determine this likelihood, we need to know 
the percentage of incoming messages that are spam, the percentage of spam messages in which 
each of these words occurs, and the percentage of messages that are not spam in which each of 
these words occurs. 

The result that we can use to answer questions such as these is called Bayes’ theorem and 
dates back to the eighteenth century. In the past two decades, Bayes’ theorem has been exten- 
sively applied to estimate probabilities based on partial evidence in areas as diverse as medicine, 
law, machine learning, engineering, and software development. 


7.3.2 Bayes’ Theorem 


We illustrate the idea behind Bayes’ theorem with an example that shows that when extra infor- 
mation is available, we can derive a more realistic estimate that a particular event occurs. That 
is, suppose we know p(F), the probability that an event F occurs, but we have knowledge that 
an event E occurs. Then the conditional probability that F occurs given that E occurs, p(F | E), 
is a more realistic estimate than p(F) that F occurs. In Example 1 we will see that we can find 


D(F | E) when we know p(F), p(E | F), and p(E | F). 


EXAMPLE 1 


Extra » 
Examples 


We have two boxes. The first contains two green balls and seven red balls; the second contains 
four green balls and three red balls. Bob selects a ball by first choosing one of the two boxes at 
random. He then selects one of the balls in this box at random. If Bob has selected a red ball, 
what is the probability that he selected a ball from the first box? 


Solution: Let E be the event that Bob has chosen a red ball; E is the event that Bob has chosen a 
green ball. Let F be the event that Bob has chosen a ball from the first box; F is the event that Bob 
has chosen a ball from the second box. We want to find p(F | E), the probability that the ball Bob 
selected came from the first box, given that it is red. By the definition of conditional probability, 
we have p(F | E) = p(F N E)/p(E). Can we use the information provided to determine both 
p(F N E) and p(E) so that we can find p(F | £)? 
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First, note that because the first box contains seven red balls out of a total of nine balls, 
we know that p(E | F) = 7/9. Similarly, because the second box contains three red balls 
out of a total of seven balls, we know that p(E | F)= 3/7. We assumed that Bob selects 
a box at random, so p(F) = = p(F ) = 1/2. Because p(E | F) = p(E N F)/p(F), it follows that 

p(E N F) = p(E | F)p(F) = ios = Z [as we remarked earlier, this is one of the quantities we 
need to find to determine nF | E). Similarly, because p(E | F) = pE N F) / pF ), it follows that 
pENnF) = pE | F)p(F) = 

We can now find p(£). Note hath = - (E NFU(EN F), where EN F and E n F are disjoint 
sets. (If x belongs to both E N F and E n F, then x belongs to both F and F, which is impossible.) 
It follows that 

7 3 49 7 27 76 _ 38 


E) = p(ENF EnF) = Se So 4+4 See. 
BES PE NE)TRE OI ia toe toe 16 68 


We have now found both p(F Nn E) = 7/18 and p(E) = 38/63. We conclude that 


CE BE 2 20g, 


PPE) = a = 38/63 76 


Before we had any extra information, we assumed that the probability that Bob selected the first 
box was 1/2. However, with the extra information that the ball selected at random is red, this 
probability has increased to approximately 0.645. That is, the probability that Bob selected a 
ball from the first box increased from 1/2, when no extra information was available, to 0.645 
once we knew that the ball selected was red. < 


Using the same type of reasoning as in Example 1, we can find the conditional probability 


that an event F occurs, given that an event E has occurred, when we know p(E | F), p(E | F), 
and p(F). The result we can obtain is called Bayes’ theorem; it is named after Thomas Bayes, 
an eighteenth-century British mathematician and minister who introduced this result. 


BAYES’ THEOREM Suppose that E and F are events from a sample space S such that 


D(E) + 0 and p(F) + 0. Then 
pF | E)= eee 
P(E | F)p(F) + pE | F)p(F) 


Proof: The definition of conditional probability tells us that p(F | E) = p(E N F)/p(E) 
and p(E|F)=p(ENF)/p(F). Therefore, p(ENF)=p(F|E)p(E) and p(EnF)= 
D(E | F)p(F). Equating these two expressions for p(E N F) shows that 


DF | E)p(E) = p(E | F)p(F). 
Dividing both sides by p(E), we find that 


(E | F)p(F) 
pF p= 
P(E) 
Next, we show that p(E)= p(E | F)p(F)+ p(E | F)p(F). To see this, first note 
that E=ENS=EN(FUF) =(ENFJVENF)._ Furthermore, EAF and EAF 
are disjoint, because if x © ENF and xE ENF, then xE FAF =Ø. Consequently, 
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Links » 


EXAMPLE 2 


D(E) = p(ENF)+p(En F). We have already — shown that p(ENF)=p(E | F)p(F). 
Moreover, we have p(E|F)=p(ENF)/p(F), which shows that p(ENF)= 
D(E | F)p(F). It now follows that 


P(E) = pE N F) + pE n F) = p(E | F)p(F) + p(E | F)p(F). 


To complete the proof we insert this expression for p(£) into the equation p(F | E) = 
D(E | F)p(F)/p(£). We have proved that 


D(E | F)p(F) 


pF | E) = ———.. 
DE | F)p(F) + p(E | F)p(F) 


< 


APPLYING BAYES’ THEOREM Bayes’ theorem can be used to solve problems that arise in 
many disciplines. Next, we will discuss an application of Bayes’ theorem to medicine. In partic- 
ular, we will illustrate how Bayes’ theorem can be used to assess the probability that someone 
testing positive for a disease actually has this disease. The results obtained from Bayes’ theorem 
are often somewhat surprising, as Example 2 shows. 


Suppose that one person in 100,000 has a particular rare disease for which there is a fairly 
accurate diagnostic test. This test is correct 99.0% of the time when given to a person selected 
at random who has the disease; it is correct 99.5% of the time when given to a person selected 
at random who does not have the disease. Given this information, can we find 


(a) the probability that a person who tests positive for the disease has the disease? 
(b) the probability that a person who tests negative for the disease does not have the disease? 


Should a person who tests positive be very concerned that he or she has the disease? 


Solution: (a) Let F be the event that a person selected at random has the disease, and let E be 
the event that a person selected at random tests positive for the disease. We want to compute 
D(F | E). To use Bayes’ theorem to compute p(F | E) we need to find p(E | F), p(E | F), D(F), 
and p(F). 

We know that one person in 100,000 has this disease, so p(F) = 1/100,000 = 0.00001 and 
p(F) = 1 — 0.00001 = 0.99999. Because a person who has the disease tests positive 99% of the 
time, we know that p(E | F) = 0.99; this is the probability of a true positive, that a person with 
the disease tests positive. It follows that pE | F) = 1 — p(E | F) = 1 — 0.99 = 0.01; this is the 
probability of a false negative, that a person who has the disease tests negative. 

Furthermore, because a person who does not have the disease tests negative 99.5% of the 
time, we know that p(E | F) = 0.995. This is the probability of a true negative, that a person 
without the disease tests negative. Finally, we see that p(E | Fy=1- P(E | F) = 1 — 0.995 = 
0.005; this is the probability of a false positive, that a person without the disease tests positive. 

The probability that a person who tests positive for the disease actually has the disease is 
D(F | E). By Bayes’ theorem, we know that 


PE | F)p(F) 


DF | £) = —-TT—+£+_——_ 
D(E | F)p(F) + p(E | F)p(F) 
(0.99)(0.00001) 


= —_______________w (),002. 
(0.99)(0.00001) + (0.005)(0.99999) 


THEOREM 2 


Links » 


The use of the word 

spam for unsolicited 
e-mail comes from a 
Monty Python comedy 
sketch about a cafe 

where the food product 
Spam comes with 
everything regardless of 
whether customers want it. 
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(b) The probability that someone who tests negative for the disease does not have the disease is 
D(F | E). By Bayes’ theorem, we know that 


P(E | F)p(F) 
pE | F)p(F) + pt | F)p(F) 


_ (0.995)(0.99999) 
~ (0.995)(0.99999) + (0.01)(0.00001) 


pF | E) = 


= 0.9999999. 


Consequently, 99.99999% of the people who test negative really do not have the disease. 


In part (a) we showed that only 0.2% of people who test positive for the disease actually 
have the disease. Because the disease is extremely rare, the number of false positives on the 
diagnostic test is far greater than the number of true positives, making the percentage of people 
who test positive who actually have the disease extremely small. People who test positive for 
the diseases should not be overly concerned that they actually have the disease. < 


GENERALIZING BAYES’ THEOREM Note that in the statement of Bayes’ theorem, the 
events F and F are mutually exclusive and cover the entire sample space S (that is, F U F = S). 
We can extend Bayes’ theorem to any collection of mutually exclusive events that cover the 
entire sample space S, in the following way. 


GENERALIZED BAYES’ THEOREM Suppose that E is an event from a sample space 
S and that F}, F, ..., F, are mutually exclusive events such that )_, F; = S. Assume that 
p(E) # 0 and p(F,) # 0 fori = 1, 2, ...,n. Then 


pE | F)p(E) 


P= oe eee 
A Seer rE) 


We leave the proof of this generalized version of Bayes’ theorem as Exercise 17. 


7.3.3 Bayesian Spam Filters 


Most electronic mailboxes receive a flood of unwanted and unsolicited messages, known as 
spam. Because spam threatens to overwhelm electronic mail systems, a tremendous amount of 
work has been devoted to filtering it out. Some of the first tools developed for eliminating spam 
were based on Bayes’ theorem, such as Bayesian spam filters. 

A Bayesian spam filter uses information about previously seen e-mail messages to guess 
whether an incoming e-mail message is spam. Bayesian spam filters look for occurrences of 
particular words in messages. For a particular word w, the probability that w appears in a spam 
e-mail message is estimated by determining the number of times w appears in a message from 
a large set of messages known to be spam and the number of times it appears in a large set of 
messages known not to be spam. When we examine e-mail messages to determine whether they 
might be spam, we look at words that might be indicators of spam, such as “offer,” “special,” or 
“opportunity,” as well as words that might indicate that a message is not spam, such as “mom,” 
“lunch,” or “Jan” (where Jan is one of your friends). Unfortunately, spam filters sometimes fail 
to identify a spam message as spam; this is called a false negative. And they sometimes identify 
a message that is not spam as spam; this is called a false positive. When testing for spam, it is 
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important to minimize false positives, because filtering out wanted e-mail is much worse than 
letting some spam through. 

We will develop some basic Bayesian spam filters. First, suppose we have a set B of mes- 
sages known to be spam and a set G of messages known not to be spam. (For example, users 
could classify messages as spam when they examine them in their inboxes.) We next identify 
the words that occur in B and in G. We count the number of messages in the set containing 
each word to find ng(w) and ng(w), the number of messages containing the word w in the sets 
B and G, respectively. Then, the empirical probability that a spam message contains the word 
w is p(w) = ng(w)/|B|, and the empirical probability that a message that is not spam contains 
the word w is q(w) = ng(w)/|G|. We note that p(w) and g(w) estimate the probabilities that 
an incoming spam message, and an incoming message that is not spam, contain the word w, 
respectively. 

Now suppose we receive a new e-mail message containing the word w. Let S be the event 
that the message is spam. Let E be the event that the message contains the word w. The events S, 
that the message is spam, and S, that the message is not spam, partition the set of all messages. 
Hence, by Bayes’ theorem, the probability that the message is spam, given that it contains the 
word w, is 


P(E | S)p(S) 


PS | E) = >a... 
P(E | S)p(S) + pE | S)p(s) 


To apply this formula, we first estimate p(S), the probability that an incoming message is 


spam, as well as p(S), the probability that the incoming message is not spam. Without prior 
knowledge about the likelihood that an incoming message is spam, for simplicity we assume 
that the message is equally likely to be spam as it is not to be spam. That is, we assume that 


p(S) = p(S) = 1/2. Using this assumption, we find that the probability that a message is spam, 
given that it contains the word w, is 


P(E |S) 


p(S | E) = —————__.. 
P(E |S) + p(E |S) 


Links » 


THOMAS BAYES (1702-1761) Thomas Bayes was the son of a minister in a religious sect known as the 
Nonconformists. This sect was considered heretical in eighteenth-century Great Britain. Because of the secrecy 
of the Nonconformists, little is known of Thomas Bayes’ life. When Thomas was young, his family moved 
to London. Thomas was likely educated privately; Nonconformist children generally did not attend school. In 
1719 Bayes entered the University of Edinburgh, where he studied logic and theology. He was ordained as a 
Nonconformist minister like his father and began his work as a minister assisting his father. In 1733 he became 
minister of the Presbyterian Chapel in Tunbridge Wells, southeast of London, where he remained minister until 
1752. 

Bayes is best known for his essay on probability published in 1764, three years after his death. This essay 
was sent to the Royal Society by a friend who found it in the papers left behind when Bayes died. In the 
introduction to this essay, Bayes stated that his goal was to find a method that could measure the probability that an event happens, 
assuming that we know nothing about it, but that, under the same circumstances, it has happened a certain proportion of times. 
Bayes’ conclusions were accepted by the great French mathematician Laplace but were later challenged by Boole, who questioned 
them in his book Laws of Thought. Since then Bayes’ techniques have been subject to controversy. 

Bayes also wrote an article that was published posthumously: “An Introduction to the Doctrine of Fluxions, and a Defense of 
the Mathematicians Against the Objections of the Author of The Analyst,” which supported the logical foundations of calculus. 
Bayes was elected a Fellow of the Royal Society in 1742, with the support of important members of the Society, even though at 
that time he had no published mathematical works. Bayes’ sole known publication during his lifetime was allegedly a mystical book 
entitled Divine Benevolence, discussing the original causation and ultimate purpose of the universe. Although the book is commonly 
attributed to Bayes, no author’s name appeared on the title page, and the entire work is thought to be of dubious provenance. 
Evidence for Bayes’ mathematical talents comes from a notebook that was almost certainly written by Bayes, which contains much 
mathematical work, including discussions of probability, trigonometry, geometry, solutions of equations, series, and differential 
calculus. There are also sections on natural philosophy, in which Bayes looks at topics that include electricity, optics, and celestial 
mechanics. Bayes is also the author of a mathematical publication on asymptotic series, which appeared after his death. 


Courtesy of Stephen Stigler 
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(Note that if we have some empirical data about the ratio of spam messages to messages that 
are not spam, we can change this assumption to produce a better estimate for p(S) and for pS); 
see Exercise 22.) 

Next, we estimate p(E | S), the conditional probability that the message contains the 
word w given that the message is spam, by p(w). Similarly, we estimate p(E | S), the con- 
ditional probability that the message contains the word w, given that the message is not 
spam, by q(w). Inserting these estimates for p(E | S) and p(E | S) tells us that p(S | E) can be 
estimated by 


pw). 
rw) = 2 _, 
pw) + q(w) 
that is, r(w) estimates the probability that the message is spam, given that it contains the 
word w. If r(w) is greater than a threshold that we set, such as 0.9, then we classify the message 
as spam. 


Suppose that we have found that the word “Rolex” occurs in 250 of 2000 messages known to 
be spam and in 5 of 1000 messages known not to be spam. Estimate the probability that an 
incoming message containing the word “Rolex” is spam, assuming that it is equally likely that 
an incoming message is spam or not spam. If our threshold for rejecting a message as spam 
is 0.9, will we reject such messages? 


Solution: We use the counts that the word “Rolex” appears in spam messages and messages 
that are not spam to find that p(Rolex) = 250/2000 = 0.125 and g(Rolex) = 5/1000 = 0.005. 
Because we are assuming that it is equally likely for an incoming message to be spam as it is 
not to be spam, we can estimate the probability that an incoming message containing the word 
“Rolex” is spam by 


Rol 
p(Rolex) + (Rolex) 0.125 + 0.005 0.130 
Because r(Rolex) is greater than the threshold 0.9, we reject such messages as spam. < 


Detecting spam based on the presence of a single word can lead to excessive false posi- 
tives and false negatives. Consequently, spam filters look at the presence of multiple words. 
For example, suppose that the message contains the words w, and w,. Let E, and E, denote 
the events that the message contains the words w, and w,, respectively. To make our computa- 
tions simpler, we assume that E, and £E, are independent events and that E, | S and E, | S are 
independent events and that we have no prior knowledge regarding whether or not the message 
is spam. (The assumptions that E£} and E, are independent and that E£, | S and E, | S are inde- 
pendent may introduce some error into our computations, we assume that this error is small.) 
Using Bayes’ theorem and our assumptions, we can show (see Exercise 23) that p(S | E, N E3), 
the probability that the message is spam given that it contains both w, and wy, is 


P(E, | S)p(E | S) 


PCS | E 0 Ey) = —_—_———_———_.. 
P(E, | OPE; | S) + pE | S)p(E | S) 


We estimate the probability p(S | E, N E,) by 


PW )p2) 


r(wy, w2) = ———— a. 
P(wi)p(w2) + dw gw) 
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EXAMPLE 4 


Bayesian poisoning, the 
insertion of extra words 
to defeat spam filters, 
can use random or 
purposefully selected 
words. 


That is, r(w,, w2) estimates the probability that the message is spam, given that it contains the 
words w, and w,. When r(w,, w2) is greater than a preset threshold, such as 0.9, we determine 
that the message is likely spam. 


Suppose that we train a Bayesian spam filter on a set of 2000 spam messages and 1000 messages 
that are not spam. The word “stock” appears in 400 spam messages and 60 messages that are not 
spam, and the word “undervalued” appears in 200 spam messages and 25 messages that are not 
spam. Estimate the probability that an incoming message containing both the words “stock” and 
“undervalued” is spam, assuming that we have no prior knowledge about whether it is spam. 
Will we reject such messages as spam when we set the threshold at 0.9? 


Solution: Using the counts of each of these two words in messages known to be spam 
or known not to be spam, we obtain the following estimates: p(stock) = 400/2000 = 0.2, 
q(stock) = 60/1000 = 0.06, p(undervalued) = 200/2000 = 0.1, and  q(undervalued) = 
25/1000 = 0.025. Using these probabilities, we can estimate the probability that the message 
is spam by 


p(stock)p(undervalued) 
r(stock, undervalued) = 2A 
p(stock)p(undervalued) + g(stock)g(undervalued) 


(0.2)(0.1) 


(0.2)(0.1) + (0.06)(0.025) 


Because we have set the threshold for rejecting messages at 0.9, such messages will be rejected 
by the filter. < 


The more words we use to estimate the probability that an incoming mail message is spam, 
the better is our chance that we correctly determine whether it is spam. In general, if E; is the 
event that the message contains word w,, assuming that the number of incoming spam messages 
is approximately the same as the number of incoming messages that are not spam, and that the 
events E; | S are independent, then by Bayes’ theorem the probability that a message containing 
all the words w,, w3, ..., W, iS spam is 


k k 
t pE;|S) 
DS | N E) = e 
i=l ILa pE | S) + IH- P(E; | $) 


We can estimate this probability by 


o Marw 
Th pwi) + Th q(w;) 


1(W1, Wo, .-., Wp) = 

For the most effective spam filter, we choose words for which the probability that each 
of these words appears in spam is either very high or very low. When we compute this value 
for a particular message, we reject the message as spam if r(w), W2, ..., W,) exceeds a preset 
threshold, such as 0.9. 

Another way to improve the performance of a Bayesian spam filter is to look at the prob- 
abilities that particular pairs of words appear in spam and in messages that are not spam. We 
then treat appearances of these pairs of words as appearance of a single block, rather than as 
the appearance of two separate words. For example, the pair of words “enhance performance” 
most likely indicates spam, while “operatic performance” indicates a message that is not spam. 
Similarly, we can assess the likelihood that a message is spam by examining the structure of a 
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message to determine where words appear in it. Also, spam filters look at appearances of cer- 
tain types of strings of characters rather than just words. For example, a message with the valid 
e-mail address of one of your friends is less likely to be spam (if not sent by a worm) than one 
containing an e-mail address that came from a country known to originate a lot of spam. There 
is an ongoing war between people who create spam and those trying to filter their messages out. 
This leads to the introduction of many new techniques to defeat spam filters, including inserting 
into spam messages long strings of words that appear in messages that are not spam, as well as 
including words inside pictures. The techniques we have discussed here are only the first steps 


in fighting this war on spam. 


Exercises 


. Suppose that E and F are events in a sample space 
and p(E) = 1/3, p(F) = 1/2, and p(E | F) = 2/5. Find 
D(F | E). 
. Suppose that E and F are events in a sample space and 
D(E) = 2/3, p(F) = 3/4, and p(F | E) = 5/8. Find p(E | 
F). 
. Suppose that Frida selects a ball by first picking one of 
two boxes at random and then selecting a ball from this 
box at random. The first box contains two white balls and 
three blue balls, and the second box contains four white 
balls and one blue ball. What is the probability that Frida 
picked a ball from the first box if she has selected a blue 
ball? 
. Suppose that Ann selects a ball by first picking one of two 
boxes at random and then selecting a ball from this box. 
The first box contains three orange balls and four black 
balls, and the second box contains five orange balls and 
six black balls. What is the probability that Ann picked 
a ball from the second box if she has selected an orange 
ball? 
. Suppose that 8% of all bicycle racers use steroids, that 
a bicyclist who uses steroids tests positive for steroids 
96% of the time, and that a bicyclist who does not use 
steroids tests positive for steroids 9% of the time. What 
is the probability that a randomly selected bicyclist who 
tests positive for steroids actually uses steroids? 
. When a test for steroids is given to soccer players, 98% 
of the players taking steroids test positive and 12% of the 
players not taking steroids test positive. Suppose that 5% 
of soccer players take steroids. What is the probability 
that a soccer player who tests positive takes steroids? 
. Suppose that a test for opium use has a 2% false positive 
rate and a 5% false negative rate. That is, 2% of people 
who do not use opium test positive for opium, and 5% of 
opium users test negative for opium. Furthermore, sup- 
pose that 1% of people actually use opium. 
a) Find the probability that someone who tests negative 
for opium use does not use opium. 
b) Find the probability that someone who tests positive 
for opium use actually uses opium. 
. Suppose that one person in 10,000 people has a rare ge- 
netic disease. There is an excellent test for the disease; 
99.9% of people with the disease test positive and only 
0.02% who do not have the disease test positive. 


10. 


11. 


a) What is the probability that someone who tests posi- 
tive has the genetic disease? 


b) What is the probability that someone who tests nega- 
tive does not have the disease? 


. Suppose that 8% of the patients tested in a clinic are in- 


fected with HIV. Furthermore, suppose that when a blood 

test for HIV is given, 98% of the patients infected with 

HIV test positive and that 3% of the patients not infected 

with HIV test positive. What is the probability that 

a) a patient testing positive for HIV with this test is in- 
fected with it? 


b) a patient testing positive for HIV with this test is not 
infected with it? 


c) a patient testing negative for HIV with this test is in- 
fected with it? 


d) a patient testing negative for HIV with this test is not 
infected with it? 


Suppose that 4% of the patients tested in a clinic are in- 
fected with avian influenza. Furthermore, suppose that 
when a blood test for avian influenza is given, 97% of 
the patients infected with avian influenza test positive and 
that 2% of the patients not infected with avian influenza 
test positive. What is the probability that 


a) a patient testing positive for avian influenza with this 
test is infected with it? 


b) a patient testing positive for avian influenza with this 
test is not infected with it? 


c) a patient testing negative for avian influenza with this 
test is infected with it? 


d) a patient testing negative for avian influenza with this 
test is not infected with it? 


An electronics company is planning to introduce a new 
camera phone. The company commissions a marketing 
report for each new product that predicts either the suc- 
cess or the failure of the product. Of new products in- 
troduced by the company, 60% have been successes. 
Furthermore, 70% of their successful products were pre- 
dicted to be successes, while 40% of failed products were 
predicted to be successes. Find the probability that this 
new camera phone will be successful if its success has 
been predicted. 
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13. 


14. 


15. 


16. 


A space probe near Neptune communicates with Earth 

using bit strings. Suppose that in its transmissions it sends 

a | one-third of the time and a 0 two-thirds of the time. 

When a 0 is sent, the probability that it is received cor- 

rectly is 0.9, and the probability that it is received incor- 

rectly (as a 1) is 0.1. When a | is sent, the probability that 
it is received correctly is 0.8, and the probability that it is 

received incorrectly (as a 0) is 0.2. 

a) Find the probability that a O is received. 

b) Use Bayes’ theorem to find the probability that a 0 
was transmitted, given that a 0 was received. 

Suppose that E, F,, Fj, and F} are events from 

a sample space § and that F,, F,, and F, are 

pairwise disjoint and their union is S. Find p(F; | 

E) if p(E | Fi) = 1/8, p(E | F,)=1/4, p(E | F3)=1/6, 

p(F\)= 1/4, p(F,) = 1/4, and p(F3) = 1/2. 

Suppose that E, F;, F), and F} are events from a 

sample space S and that F,, Fj, and F, are pair- 

wise disjoint and their union is S. Find p(F, | E) if 

PE |F\)=2/7, pE| Fy) =3/8, pE | F3) = 1/2, 

p(F,) = 1/6, p(F,) = 1/2, and p(F3) = 1/3. 

In this exercise we will use Bayes’ theorem to solve the 

Monty Hall puzzle (Example 10 in Section 7.1). Recall 

that in this puzzle you are asked to select one of three 

doors to open. There is a large prize behind one of the 
three doors and the other two doors are losers. After you 
select a door, Monty Hall opens one of the two doors 
you did not select that he knows is a losing door, se- 
lecting at random if both are losing doors. Monty asks 
you whether you would like to switch doors. Suppose 

that the three doors in the puzzle are labeled 1, 2, and 3. 

Let W be the random variable whose value is the num- 

ber of the winning door; assume that p(W = k) = 1/3 

for k = 1, 2, 3. Let M denote the random variable whose 

value is the number of the door that Monty opens. Sup- 

pose you choose door i. 

a) What is the probability that you will win the prize if 
the game ends without Monty asking you whether you 
want to change doors? 

b) Find p(M =j | W = k) for j = 1, 2,3 and k = 1, 2, 3. 

c) Use Bayes’ theorem to find p(W = j | M = k) where i 
and j and k are distinct values. 

d) Explain why the answer to part (c) tells you whether 
you should change doors when Monty gives you the 
chance to do so. 


Ramesh can get to work in three different ways: by bicy- 
cle, by car, or by bus. Because of commuter traffic, there 
is a 50% chance that he will be late when he drives his 
car. When he takes the bus, which uses a special lane re- 
served for buses, there is a 20% chance that he will be 
late. The probability that he is late when he rides his bi- 
cycle is only 5%. Ramesh arrives late one day. His boss 
wants to estimate the probability that he drove his car to 
work that day. 
a) Suppose the boss assumes that there is a 1/3 chance 
that Ramesh takes each of the three ways he can get to 
work. What estimate for the probability that Ramesh 


*17; 


18. 


19. 


20. 


21. 


22. 


drove his car does the boss obtain from Bayes’ theo- 
rem under this assumption? 


b) Suppose the boss knows that Ramesh drives 30% of 
the time, takes the bus only 10% of the time, and takes 
his bicycle 60% of the time. What estimate for the 
probability that Ramesh drove his car does the boss 
obtain from Bayes’ theorem using this information? 

Prove Theorem 2, the extended form of Bayes’ theorem. 

That is, suppose that £E is an event from a sample space S 

and that F}, F>, ..., F, are mutually exclusive events such 

that ()'_, F; = S. Assume that p(E) # 0 and p(F;) # 0 for 
i= 1,2,...,n. Show that 


p(E | F)p(F) 
Xip E l F,)p(F;) 


DF; | E) = 


[Hint: Use the fact that E = | Ji (En F)).] 


Suppose that a Bayesian spam filter is trained on a set 
of 500 spam messages and 200 messages that are not 
spam. The word “exciting” appears in 40 spam messages 
and in 25 messages that are not spam. Would an in- 
coming message be rejected as spam if it contains the 
word “exciting” and the threshold for rejecting spam 
is 0.9? 


Suppose that a Bayesian spam filter is trained on a set 
of 1000 spam messages and 400 messages that are not 
spam. The word “opportunity” appears in 175 spam mes- 
sages and 20 messages that are not spam. Would an in- 
coming message be rejected as spam if it contains the 
word “opportunity” and the threshold for rejecting a mes- 
sage is 0.9? 


Would we reject a message as spam in Example 4 


a) using just the fact that the word “undervalued” occurs 
in the message? 


b) using just the fact that the word “stock” occurs in the 
message? 

Suppose that a Bayesian spam filter is trained on a set 
of 10,000 spam messages and 5000 messages that are 
not spam. The word “enhancement” appears in 1500 
spam messages and 20 messages that are not spam, while 
the word “herbal” appears in 800 spam messages and 
200 messages that are not spam. Estimate the probabil- 
ity that a received message containing both the words 
“enhancement” and “herbal” is spam. Will the message 
be rejected as spam if the threshold for rejecting spam 
is 0.9? 


Suppose that we have prior information concerning 

whether a random incoming message is spam. In par- 

ticular, suppose that over a time period, we find that s 

spam messages arrive and h messages arrive that are 

not spam. 

a) Use this information to estimate p(S), the probabil- 
ity that an incoming message is spam, and p(S), the 
probability an incoming message is not spam. 
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b) Use Bayes’ theorem and part (a) to estimate the prob- E, | S and E, | S are independent events, where S is the 
ability that an incoming message containing the word event that an incoming message is spam, and that we have 
w is spam, where p(w) is the probability that w occurs no prior knowledge regarding whether or not the message 
in a spam message and q(w) is the probability that w is spam, show that 
occurs in a message that is not spam. 

P(S | Ei NE) 
23. Suppose that E£; and Æ, are the events that an incoming 
mail message contains the words w, and wy, respectively. = PE | S)p(E | a = 
Assuming that E£, and E, are independent events and that P(E, | S)p(E, | S) + pE | pE, | S) 


Expected Value and Variance 


Links > 


Definition 1 


7.4.1 Introduction 


The expected value of a random variable is the sum over all elements in a sample space of the 
product of the probability of the element and the value of the random variable at this element. 
Consequently, the expected value is a weighted average of the values of a random variable. The 
expected value of a random variable provides a central point for the distribution of values of 
this random variable. We can solve many problems using the notion of the expected value of a 
random variable, such as determining who has an advantage in gambling games and computing 
the average-case complexity of algorithms. Another useful measure of a random variable is its 
variance, which tells us how spread out the values of this random variable are. We can use the 
variance of a random variable to help us estimate the probability that a random variable takes 
values far removed from its expected value. 


7.4.2 Expected Values 


Many questions can be formulated in terms of the value we expect a random variable to take, 
or more precisely, the average value of a random variable when an experiment is performed a 
large number of times. Questions of this kind include: How many heads are expected to appear 
when a coin is flipped 100 times? What is the expected number of comparisons used to find an 
element in a list using a linear search? To study such questions we introduce the concept of the 
expected value of a random variable. 


The expected value, also called the expectation or mean, of the random variable X on the 
sample space S is equal to 


EX) = Ý pX). 
ses 


The deviation of X at s E€ S is X(s) — E(X), the difference between the value of X and the 
mean of X. 


Note that when the sample space S has n elements S = {x}, x, ...,.x,}, EX) = YL, PX). 


Remark: When there are infinitely many elements of the sample space, the expectation is de- 
fined only when the infinite series in the definition is absolutely convergent. In particular, the 
expectation of a random variable on an infinite sample space is finite if it exists. 
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EXAMPLE 1 


EXAMPLE 2 


Extra > 
Examples 


THEOREM 1 


Expected Value of a Die Let X be the number that comes up when a fair die is rolled. What 
is the expected value of X? 


Solution: The random variable X takes the values 1, 2, 3, 4, 5, or 6, each with probability 1/6. 
It follows that 


1 1 1 1 7 
EQQ)aS-14 2-24 22394 S54 e254 5062225, 
X) A 3+3 Hoest 5 < 


A fair coin is flipped three times. Let S be the sample space of the eight possible outcomes, and 
let X be the random variable that assigns to an outcome the number of heads in this outcome. 
What is the expected value of X? 


Solution: In Example 10 of Section 7.2 we listed the values of X for the eight possible outcomes 


when a coin is flipped three times. Because the coin is fair and the flips are independent, the 
probability of each outcome is 1/8. Consequently, 


E(X) = LIX(HHH) + X(HHT) + X(HTH) + X(THH) + X(TTH) 


+X(THT) + X(HTT) + X(TTT)| 
={G+2+2+2+1+1+1+0)= = 


Consequently, the expected number of heads that come up when a fair coin is flipped three times 


is 3/2. < 


When an experiment has relatively few outcomes, we can compute the expected value of 
a random variable directly from its definition, as was done in Example 2. However, when an 
experiment has a large number of outcomes, it may be inconvenient to compute the expected 
value of a random variable directly from its definition. Instead, we can find the expected value 
of a random variable by grouping together all outcomes assigned the same value by the random 
variable, as Theorem 1 shows. 


If X is a random variable and p(X = r) is the probability that X = r, so that p(X =r) = 
È ses x= PCS), then 


E(X) = D p(X = ryr. 


rEX(S) 


Proof: Suppose that X is a random variable with range X(S), and let p(X = r) be the proba- 
bility that the random variable X takes the value r. Consequently, p(X = r) is the sum of the 
probabilities of the outcomes s such that X(s) = r. It follows that 


E(X) = > p(X = ryr. < 


rEX(S) 
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Example 3 and the proof of Theorem 2 will illustrate the use of this formula. In Example 
3 we will find the expected value of the sum of the numbers that appear on two fair dice when 
they are rolled. In Theorem 2 we will find the expected value of the number of successes when 
n Bernoulli trials are performed. 


EXAMPLE 3 What is the expected value of the sum of the numbers that appear when a pair of fair dice is 
rolled? 


Solution: Let X be the random variable equal to the sum of the numbers that appear when a 
pair of dice is rolled. In Example 12 of Section 7.2 we listed the value of X for the 36 out- 
comes of this experiment. The range of X is {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}. By Example 12 of 
Section 7.2 we see that 


P(X = 2) = p(X = 12) = 1/36, 
p(X = 3) = p(X = 11) = 2/36 = 1/18, 
p(X = 4) = p(X = 10) = 3/36 = 1/12, 
D(X = 5) = p(X = 9) = 4/36 = 1/9, 
p(X = 6) = p(X = 8) = 5/36, 

p(X = 7) = 6/36 = 1/6. 


Substituting these values in the formula, we have 


l l 1 l 5 l 
ROHS 3g E E EEE E, EE Se 
NSE sg ge ig a a a 
5 1 1 l 1 
mee EE ay (| Pee eG E | E 
Aa a en: la 

ak < 


THEOREM 2 The expected number of successes when n mutually independent Bernoulli trials are per- 
formed, where p is the probability of success on each trial, is np. 


Proof: Let X be the random variable equal to the number of successes in n trials. By 
Theorem 2 of Section 7.2 we see that p(X = k) = C(n, k)p‘q"~*. Hence, we have 


E(X) = 2 kp(X = k) by Theorem 1 
k=1 
= kC(n, kpg" by Theorem 2 in Section 7.2 
k=1 
= > nC(n— 1, k — 1)p*q""* by Exercise 21 in Section 6.4 
k=1 


n 
= np > C(n— 1, k —1)p*"'q"* factoring np from each term 
k=1 
n-1 
= np Cin-1, jp! guid shifting index of summation with j = k — 1 
j=0 
= np(p + gy" by the binomial theorem 


=np because p+qg=1. 
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THEOREM 3 


EXAMPLE 4 


EXAMPLE 5 


This completes the proof because it shows that the expected number of successes in n mutually 
independent Bernoulli trials is np. < 


We will also show that the hypothesis that the Bernoulli trials are mutually independent in 
Theorem 2 is not necessary. 


7.4.3 Linearity of Expectations 


Theorem 3 tells us that expected values are linear. For example, the expected value of the sum 
of random variables is the sum of their expected values. We will find this property exceedingly 
useful. 


If X, i= 1,2, ...,n with n a positive integer, are random variables on S, and if a and b are 
real numbers, then 


(i) EG +X, T a +X,) = E E) e + EX,) 
(ii) E(aX + b) = aE(X) + b. 


Proof: Part (i) follows for n = 2 directly from the definition of expected value, because 


EX, +X) = È POZE) +X) 


ses 


¥Y PIX (3) + È p()Xy(s) 


ses ses 


= E(X,) + E(X). 


The case for n random variables follows easily by mathematical induction using the case of two 
random variables. (We leave it to the reader to complete the proof.) 
To prove part (ii), note that 


E(aX + b) = $ es P(s)(aX(s) + b) 


=a Eses P(s)X(s) +b È ses PCS) 
= aE(X) + b because eṣ p(s) = 1. < 


Examples 4 and 5 illustrate how to use Theorem 3. 


Use Theorem 3 to find the expected value of the sum of the numbers that appear when a pair of 
fair dice is rolled. (This was done in Example 3 without the benefit of this theorem.) 


Solution: Let X, and X, be the random variables with X, ((ij)) =i and X,((i j)) =j, so 
that X, is the number appearing on the first die and X, is the number appearing on the second 
die. It is easy to see that E(X,) = E(X,) = 7/2 because both equal (1 +2 +3 +4 +5 + 6)/6 = 
21/6 = 7/2. The sum of the two numbers that appear when the two dice are rolled is the 
sum X; + X,. By Theorem 3, the expected value of the sum is E(X, + X2) = E(X,;) + E(X2) = 
7/24+7/2 =7. < 


In the proof of Theorem 2 we found the expected value of the number of successes when n 
independent Bernoulli trials are performed, where p is the probability of success on each trial by 


EXAMPLE 6 


EXAMPLE 7 
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direct computation. Show how Theorem 3 can be used to derive this result where the Bernoulli 
trials are not necessarily independent. 


Solution: Let X; be the random variable with X,((t,,,...,4,))=1 if t; is a suc- 
cess and X,((t,h,...,4,)) =0 if t; is a failure. The expected value of X, is E(X) = 
1-p+0-(_—-p)=p for 1=1,2,...,n. Let X=X,+X,+:+::+X,, so that X counts 
the number of successes when these n Bernoulli trials are performed. Theorem 3, 
applied to the sum of n random variables, shows that E(X) = E(X,)+ E(X,) +--+ 

< 


E(X,,) = np. 


We can take advantage of the linearity of expectations to find the solutions of many seem- 
ingly difficult problems. The key step is to express a random variable whose expectation we 
wish to find as the sum of random variables whose expectations are easy to find. Examples 6 
and 7 illustrate this technique. 


Expected Value in the Hatcheck Problem A new employee checks the hats of n people at a 
restaurant, forgetting to put claim check numbers on the hats. When customers return for their 
hats, the checker gives them back hats chosen at random from the remaining hats. What is the 
expected number of hats that are returned correctly? 


Solution: Let X be the random variable that equals the number of people who receive the correct 
hat from the checker. Let X, be the random variable with X; = 1 if the ith person receives the 
correct hat and X; = 0 otherwise. It follows that 


X=X +X% oe +X, 


Because it is equally likely that the checker returns any of the hats to this person, it follows that 
the probability that the ith person receives the correct hat is 1/n. Consequently, by Theorem 1, 
for all i we have 


E(X;) = 1- p(X; = 1) + 0- p(X, = 0) = 1-1/n+0= 1/n. 
By the linearity of expectations (Theorem 3), it follows that 
E(X) = E(X) + F(X) + + F(X) =n-1/n=1. 


Consequently, the average number of people who receive the correct hat is exactly 1. Note 
that this answer is independent of the number of people who have checked their hats! 
(We will find an explicit formula for the probability that no one receives the correct hat in 
Example 4 of Section 8.6.) < 


Expected Number of Inversions in a Permutation The ordered pair (i,j) is called an 
inversion in a permutation of the first n positive integers if i < j but j precedes i in the 
permutation. For instance, there are six inversions in the permutation 3, 5, 1, 4, 2; these 
inversions are 


(1, 3), (1, 5), (2, 3), (2, 4, (2, 5), (4 5). 


Let J; ; be the random variable on the set of all permutations of the first n positive integers with 
IL; = 1 if @ j) is an inversion of the permutation and J; ; = 0 otherwise. It follows that if X is the 
random variable equal to the number of inversions in the permutation, then 

X= 5 A 


iy 
l<i<j<n 
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Links » 


EXAMPLE 8 


Note that it is equally likely for i to precede j in a randomly chosen permutation as it is for j to 
precede i. (To see this, note that there are an equal number of permutations with each of these 
properties.) Consequently, for all pairs i and j we have 


E) = 1+ pl, = )+0- pd, = 0) = 1-1/2+0= 1/2. 


n 


Because there are ( 


orem 3), we have 


-1 
EQ = J Edi) = (°) l ; = E 


1<i<j<n 


) pairs i and j with 1 < i < j < n and by the linearity of expectations (The- 


It follows that there are an average of n(n — 1)/4 inversions in a permutation of the first n positive 
integers. < 


7.4.4 Average-Case Computational Complexity 


Computing the average-case computational complexity of an algorithm can be interpreted as 
computing the expected value of a random variable. Let the sample space of an experiment be 
the set of possible inputs aj, j= 1,2, ..., n, and let X be the random variable that assigns to a, the 
number of operations used by the algorithm when given a; as input. Based on our knowledge of 
the input, we assign a probability p(a;) to each possible input value a;. Then, the average-case 
complexity of the algorithm is 


n 


EX) = J, pa )Xa). 


j=l 


This is the expected value of X. 

Finding the average-case computational complexity of an algorithm is usually much more 
difficult than finding its worst-case computational complexity, and often involves the use of 
sophisticated methods. However, there are some algorithms for which the analysis required 
to find the average-case computational complexity is not difficult. For instance, in Example 8 
we will illustrate how to find the average-case computational complexity of the linear search 
algorithm under different assumptions concerning the probability that the element for which we 
search is an element of the list. 


Average-Case Complexity of the Linear Search Algorithm We are given a real number x 
and a list of n distinct real numbers. The linear search algorithm, described in Section 3.1, 
locates x by successively comparing it to each element in the list, terminating when x is located 
or when all the elements have been examined and it has been determined that x is not in the 
list. What is the average-case computational complexity of the linear search algorithm if the 
probability that x is in the list is p and it is equally likely that x is any of the n elements in 
the list? (There are n + 1 possible types of input: one type for each of the n numbers in the list 
and a last type for numbers not in the list, which we treat as a single input.) 


Solution: In Example 4 of Section 3.3 we showed that 2i + 1 comparisons are used if x equals 
the ith element of the list and, in Example 2 of Section 3.3, we showed that 2n + 2 comparisons 
are used if x is not in the list. The probability that x equals a;, the ith element in the list, is 


EXAMPLE 9 
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p/n, and the probability that x is not in the list is q = 1 — p. It follows that the average-case 
computational complexity of the linear search algorithm is 


3 5 2n+1 
Bay OP 


n 


+ (2n+ 2)q 


= 2G 4+54--+(Qn+1)) + Qn+2q 


= F(n+ 1? - I) + 2n + 24 


= p(n + 2) + (2n + 2)q. 


(The third equality follows from Example 2 of Section 5.1.) For instance, when x is guaranteed 
to be in the list, we have p = 1 (so the probability that x = a; is 1/n for each i) and q = 0. Then 
E = n +2, as we showed in Example 4 in Section 3.3. 

When p, the probability that x is in the list, is 1/2, it follows that q = 1 — p = 1/2, 
so E=(n+2)/2+n+1 = (3n+4)/2. Similarly, if the probability that x is in the list 
is 3/4, we have p = 3/4 and q = 1/4, so E = 3(n + 2)/4 + (n + 1)/2 = (5n + 8)/4. 

Finally, when x is guaranteed not to be in the list, we have p = 0 and q = 1. It follows that 
E = 2n + 2, which is not surprising because we have to search the entire list. < 


Example 9 illustrates how the linearity of expectations can help us find the average-case 
complexity of a sorting algorithm, the insertion sort. 


Average-Case Complexity of the Insertion Sort What is the average number of comparisons 
used by the insertion sort to sort n distinct elements? 


Solution: We first suppose that X is the random variable equal to the number of comparisons 
used by the insertion sort (described in Section 3.1) to sort a list ay, a), ..., a, of n distinct 
elements. Then E(X) is the average number of comparisons used. (Recall that at step i for i = 
2, ..., n, the insertion sort inserts the ith element in the original list into the correct position in 
the sorted list of the first i— 1 elements of the original list.) 

We let X; be the random variable equal to the number of comparisons used to insert a; into 


the proper position after the first i— 1 elements a), a, ..., a;_; have been sorted. Because 
X =X, +X; + +X, 

we can use the linearity of expectations to conclude that 
E(X) = E(X, + X; + + + X,,) = E(X2) + E(X3) + © + E(X,). 


To find E(X;) for i= 2,3,...,n, let p(k) denote the probability that the largest of the 
first j elements in the list occurs at the kth position, that is, that max(a}, dp, -> 4;) = dp 
where 1 < k < j. Because the elements of the list are randomly distributed, it is equally likely 
for the largest element among the first j elements to occur at any position. Consequently, 
p(k) = 1/j. If X(k) equals the number of comparisons used by the insertion sort if a; is 
inserted into the kth position in the list once a,,d,...,a;_,; have been sorted, it follows 
that X,(k) = k. Because it is possible that a; is inserted in any of the first i positions, we 
find that 


i i 


EX) = J) pik) X=} > k= 


k=1 k=1 k=1 


1 i@+1)_ i+] 


i 2 2 


> 
Il 
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It follows that 


n n n+l 
_ _ i+1_ 1 . 
EX) =J EX) = EA 
i=2 i=2 J3 
2 = 
a Lee D2) Ne oye + 3n a 
2 2 2 4 


To obtain the third of these equalities we shifted the index of summation, setting j = i+ 1. 
To obtain the fourth equality, we used the formula }Y”"_, k = m(m + 1)/2 (from Table 2 in Sec- 
tion 2.4) with m =n + 1, subtracting off the missing terms with j = 1 and j = 2. We conclude 
that the average number of comparisons used by the insertion sort to sort n elements equals 
(n? + 3n — 4)/4, which is @(n’). < 


7.4.5 The Geometric Distribution 


We now turn our attention to a random variable with infinitely many possible outcomes. 


Suppose that the probability that a coin comes up tails is p. This coin is flipped repeatedly until 
it comes up tails. What is the expected number of flips until this coin comes up tails? 


Solution: We first note that the sample space consists of all sequences that begin with any 
number of heads, denoted by H, followed by a tail, denoted by T. Therefore, the sample 
space is the set {7} HT, HHT, HHHT, HHHHT, ...}. Note that this is an infinite sample space. 
We can determine the probability of an element of the sample space by noting that the 
coin flips are independent and that the probability of a head is 1 — p. Therefore, p(T) = p, 
p(HT) = (1 — p)p, p(HHT) = (1 — p)’p, and in general the probability that the coin is flipped n 
times before a tail comes up, that is, that n — 1 heads come up followed by a tail, is (1 — p)""!p. 
(Exercise 14 asks for a verification that the sum of the probabilities of the points in the sample 
space is 1.) 

Now let X be the random variable equal to the number of flips in an element in the 
sample space. That is, X(T) = 1, X(HT) = 2, X(HHT) = 3, and so on. Note that p(X = j) = 
(1 — py~!p. The expected number of flips until the coin comes up tails equals E(X). 

Using Theorem 1, we find that 


foe) 


EX) =) j pX =)= Vid -py'p=p 
} a 


j=l J 


S, 2 1_1 

a-p =p -= -. 

il P 

[The third equality in this chain follows from Table 2 in Section 2.4, which tells us that 
> jd — py! =1/(. —(1 — p)? = 1/p?.] It follows that the expected number of times the 
j=l P 


coin is flipped until tails comes up is 1/p. Note that when the coin is fair we have p = 1/2, so 
the expected number of flips until it comes up tails is 1/(1/2) = 2. < 


The random variable X that equals the number of flips expected before a coin comes up 
tails is an example of a random variable with a geometric distribution. 


A random variable X has a geometric distribution with parameter p if p(X = k) = 
(1 — p) !p for k = 1, 2, 3, ..., where p is a real number with 0 < p < 1. 
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Geometric distributions arise in many applications because they are used to study the time re- 
quired before a particular event happens, such as the time required before we find an object with 
a certain property, the number of attempts before an experiment succeeds, the number of times 
a product can be used before it fails, and so on. 

When we computed the expected value of the number of flips required before a coin comes 
up tails, we proved Theorem 4. 


If the random variable X has the geometric distribution with parameter p, then E(X) = 1/p. 


7.4.6 Independent Random Variables 


We have already discussed independent events. We will now define what it means for two ran- 
dom variables to be independent. 


The random variables X and Y on a sample space S are independent if 
pec wand Y=") — pC — 7) p(s); 


or in words, if the probability that X = r, and Y = r, equals the product of the probabilities 
that X = r; and Y = ry, for all real numbers r; and r,. 


Are the random variables X, and X, from Example 4 independent? 


Solution: Let S = {1, 2, 3, 4,5, 6}, and let į E€ S and j E€ S. Because there are 36 possible out- 
comes when the pair of dice is rolled and each is equally likely, we have 


p(X, = i and X, = j) = 1/36. 


Furthermore, p(X, = i) = 1/6 and p(X, = j) = 1/6, because the probability that i appears on 
the first die and the probability that j appears on the second die are both 1/6. It follows that 


1 1 1 
X, = i and X, = j) = — d X =i X =j= =.> 
p(X, =i and X, = j) 36 and p(X, = )p(X% =j) 66 36’ 


so X, and X, are independent. < 


Show that the random variables X, and X = X, + X,, where X, and X, are as defined in Exam- 
ple 4, are not independent. 


Solution: Note that p(X; = 1 and X = 12) = 0, because X, = 1 means the number appear- 
ing on the first die is 1, which implies that the sum of the numbers appearing on the two 
dice cannot equal 12. On the other hand, p(X, = 1) = 1/6 and p(X = 12) = 1/36. Hence 
p(X, = 1 and X = 12) 4 p(X, = 1) - p(X = 12). This counterexample shows that X, and X are 
not independent. < 
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The expected value of the product of two independent random variables is the product of 
their expected values, as Theorem 5 shows. 


If X and Y are independent random variables on a sample space S, then E(XY) = E(X)E(Y). 


Proof: To prove this formula, we use the key observation that the event XY = r is the disjoint 
union of the events X = r, and Y = r, over all r} € X(S) and r, E€ Y(S) with r = r,r,. We have 


E(XY) = > r+ p(XY =r) by Theorem 1 

rEXY(S) 

= 2 rir : p(X =r,and Y = r) expressing XY = r as a disjoint union 
r EXSI NEY(S) 

= > rir : p(X =r, and Y = r) using a double sum to order the terms 
ry EX(S) nEY(S) 

= > rir * p(X =r): PY =r) by the independence of X and Y 
r EX(S) nEY(S) 

= y (ri pX =r): > r, *p(Y= r2)) by factoring out r; - p(X = r,) 
r EXS) meY(S) 

= > ri -p(X =r): EY) by the definition of E(Y) 
r,EX(S) 

= E(Y)( » r pX = rı)) by factoring out E(Y) 

r,EX(S) 
= E(Y)E(X) by the definition of E(X). 


We complete the proof by noting that E(Y)E(X) = E(X)E(Y), which is a consequence of the 
commutative law for multiplication. < 


Note that when X and Y are random variables that are not independent, we cannot conclude 
that E(XY) = E(X)E(Y), as Example 13 shows. 


Let X and Y be random variables that count the number of heads and the number of tails when a 
coin is flipped twice. Because p(X = 2) = 1/4, p(X = 1) = 1/2, and p(X = 0) = 1/4, by The- 
orem 1 we have 


1 1 
EX)=2--+1-2 +0- 
©) 47"? 4 


A similar computation shows that E(Y) = 1. We note that XY = 0 when either two heads and 


no tails or two tails and no heads come up and that XY = 1 when one head and one tail come 
up. Hence, 


1 1 
E(XY) =1-=+0-==-. 
(XY) eS 5 


Nl 


It follows that 


E(XY) # E(X)E(Y). 
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This does not contradict Theorem 5 because X and Y are not independent, as the reader should 
verify (see Exercise 16). < 


7.4.7 Variance 


Links » The expected value of a random variable tells us its average value, but nothing about how widely 
its values are distributed. For example, if X and Y are the random variables on the set S$ = 
{1, 2, 3,4, 5, 6}, with X(s) = 0 for all s E€ S and Y(s) = —1 if s € {1,2,3} and Y(s)= 1 if s € 
{4, 5, 6}, then the expected values of X and Y are both zero. However, the random variable X 
never varies from 0, while the random variable Y always differs from 0 by 1. The variance of a 
random variable helps us characterize how widely a random variable is distributed. In particular, 
it provides a measure of how widely X is distributed about its expected value. 


Definition 4 Let X be a random variable on a sample space S. The variance of X, denoted by V(X), is 


V(X) = Di X6) - EXY’ p6). 


ses: 


That is, V(X) is the weighted average of the square of the deviation of X. The standard 
deviation of X, denoted o(X), is defined to be y V(X). 


Theorem 6 provides a useful simple expression for the variance of a random variable. 


THEOREM 6 If X is a random variable on a sample space S, then V(X) = E(X?) — E(X}. 


Proof: Note that 


V(X) = $ X6) - EQOY pls) 


ses 
= $ X(s)?p(s) — 2EX) F X(9)p(s) + EXY Y pls) 
ses ses ses 


= E(X*) — 2E(X)E(X) + EX? 
= E(X?) — E(x)’. 


We have used the fact that X eṣ p(s) = 1 in the next-to-last step. < 


We can use Theorems 3 and 6 to derive an alternative formula for V(X) that provides some 
insight into the meaning of the variance of a random variable. 


COROLLARY 1 IfX is a random variable on a sample space S and E(X) = p, then V(X) = E((X — w)}?). 
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Proof: If X is a random variable with E(X) = y, then 
E(X - WP) = EX? — 2X + pw) expanding (X — u)? 
= E(X?) — E(2uX) + E(u’) by part (i) of Theorem 3 


= E(X’) — 2uE(X) + E(u) by part (ii) of Theorem 3, noting that y is a constant 


= E(X?) — 24E(X) + p as E(w) = p?, because p? is a constant 
= EX) -2y? + ww? because E(X) = u 
= E(X’) - Ww simplifying 
= V(X) by Theorem 6 and noting that E(X) = p. 
This completes the proof. < 


Corollary 1 tells us that the variance of a random variable X is the expected value of the 
square of the difference between X and its own expected value. This is commonly expressed 
as saying that the variance of X is the mean of the square of its deviation. We also say that the 
standard deviation of X is the square root of the mean of the square of its deviation (often read 
as the “root mean square” of the deviation). 

We now compute the variance of some random variables. 


What is the variance of the random variable X with X(t) = 1 if a Bernoulli trial is a success and 
X(t) = 0 if it is a failure, where p is the probability of success and q is the probability of failure? 


Solution: Because X takes only the values 0 and 1, it follows that X? (£) = X(t). Hence, 


V(X) = ER) - EXP = p - P’ = p(l — p) = pq. < 


Variance of the Value of a Die What is the variance of the random variable X, where X is 
the number that comes up when a fair die is rolled? 


Solution: We have V(X) = E(X?) — E(X}. By Example 1 we know that E(X) = 7/2. To find 
E(X?) note that X? takes the values 77, i = 1, 2, ..., 6, each with probability 1/6. It follows that 


EX) = E2423? 44? +5 +6?) = a, 


We conclude that 


What is the variance of the random variable X((ji, j)) = 2i, where i is the number appearing on 
the first die and j is the number appearing on the second die, when two fair dice are rolled? 


THEOREM 7 


Links » 


©Paul Fearn/Alamy Stock 
Photo 


7.4 Expected Value and Variance 515 


Solution: We will use Theorem 6 to find the variance of X. To do so, we need to find the expected 
values of X and X?. Note that because p(X =k) is 1/6 fork = 2, 4, 6, 8, 10, 12 and is 0 otherwise, 


E(X) = (2+44+6+8+4 10+ 12)/6 =7, 
and 

E(X?) = (27 + 4? + 6? + 87 + 10? + 127)/6 = 182/3. 
It follows from Theorem 6 that 


V(X) = E(X?) — E(X}? = 182/3 — 49 = 35/3. < 


Another useful property is that the variance of the sum of two or more independent random 
variables is the sum of their variances. The formula that expresses this property is known as 
Bienaymé’s formula, after Irenée-Jules Bienaymé, the French mathematician who discovered it 
in 1853. Bienaymé’s formula is useful for computing the variance of the result of n independent 
Bernoulli trials, for instance. 


BIENAYME’S FORMULA If X and Y are two independent random variables 
on a sample space S, then V(X + Y) = V(X) + V(Y). Furthermore, if X,, i= 1,2, ...,2, 
with n a positive integer, are pairwise independent random variables on S, then 
V(X, +X, +--+ +X) = V(X) + V(X) + + + VX). 


Proof: From Theorem 6, we have 
V(X + Y) = E(X + YP) -E(X+Y/. 
It follows that 
V(X + Y) = E(X? + 2XY + Y°) — (E(X) + EY)? 
= E(X?) + 2E(XY) + EY?) — E(X)* — 2E(X)E(Y) — E(Y)’. 


IRENEE-JULES BIENAYME (1796-1878) Bienaymé, born in Paris, moved with his family to Bruges 
in 1803 when his father became a government administrator. Bienaymé attended the Lycée impérial in 
Bruges, and when his family returned to Paris in 1811, the Lycée Louis-le-Grand. As a teenager, he helped 
defend Paris during the 1814 Napoleonic Wars; in 1815, he became a student at the Ecole Polytech- 
nique. In 1816 he joined the Ministry of Finances to help support his family. In 1819, he left the civil 
service, taking a job lecturing mathematics at the Académie militaire de Saint-Cyr. Unhappy with con- 
ditions there, he soon returned to the Ministry of Finances. He attained the position of inspector gen- 
eral, remaining until 1848, when he was forced to retire when a new regime installed their own peo- 
ple. He was able to return as inspector general in 1850, but he retired a second time in 1852. In 1851 
he briefly was professor at the Sorbonne and also served as an expert statistician for Napoleon III. 


Bienaymé was one of the founders of the Société Mathématique de France, and in 1875 was its president. 

Bienaymé was noted for his ingenuity, but his papers frustrated readers by omitting important proofs. He published sparsely, 
often in obscure journals. However, he made important contributions to probability and statistics, and to their applications to the 
social sciences and to finance. Among his important contributions are the Bienaymé-Chebyshev inequality, which provides a simple 
proof of the law of large numbers, a generalization of Laplace’s least square method, and Bienaymé’s formula for the variance of a 
sum of random variables. He studied the extinction of aristocratic families, declining despite general population growth. Bienaymé 
was a skilled linguist; he translated the works of Chebyshev, a close friend, from Russian to French. It has been suggested that his 
relative obscurity results from his modesty, his lack of interest in asserting the priority of his discoveries, and the fact that his work 
was often ahead of its time. He and his brother married two sisters who were daughters of a family friend. Bienaymé and his wife 
had two sons and three daughters. 
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Because X and Y are independent, by Theorem 5 we have E(XY) = E(X)E(Y). It follows that 


V(X + Y) = (EX?) — E(X?) + (EO?) — E(Y)”) 
= V(X) + V(Y). 


We leave the proof of the case for pairwise independent random variables to the reader (Ex- 
ercise 34). Such a proof can be constructed by generalizing the proof we have given for the 
case for two random variables. Note that it is not possible to use mathematical induction in a 
straightforward way to prove the general case (see Exercise 33). < 


Find the variance and standard deviation of the random variable X whose value when two fair 
dice are rolled is X((i, j)) = i + j, where i is the number appearing on the first die and j is the 
number appearing on the second die. 


Solution: Let X, and X, be the random variables defined by X,((i,j)) =i and 
X,(@j)) =j for a roll of the dice. Then X = X, + X,, and X, and X, are independent, as 
Example 11 showed. From Theorem 7 it follows that V(X) = V(X) + V(X). A sim- 
ple computation as in Example 16, together with Exercise 29 in the Supplementary 
Exercises, tells us that V(X,) = V(X,) = 35/12. Hence, V(X) = 35/12 + 35/12 = 35/6 


and o(X) = y35/6. < 


We will now find the variance of the random variable that counts the number of successes 
when n independent Bernoulli trials are carried out. 


What is the variance of the number of successes when n independent Bernoulli trials are per- 
formed, where, on each trial, p is the probability of success and q is the probability of failure? 


Solution: Let X; be the random variable with X,((4,,%,...,¢,))=1 if trial 4 is a 
success and X,((t,,,...,¢,))=0 if trial t; is a failure. Let X =X +X% + +X. 
Then X counts the number of successes in the n trials. From Theorem 7 it follows that V(X) = 
V(X) + V(X) + +++ + V(X,). Using Example 14 we have V(X;) = pq for i = 1, 2, ..., n. It fol- 
lows that V(X) = npg. < 


PAFNUTY LVOVICH CHEBYSHEV (1821-1894) Chebyshev was born into the gentry in Okatovo, Russia. 
His father was a retired army officer who had fought against Napoleon. In 1832 the family, with its nine children, 
moved to Moscow, where Pafnuty completed his high school education at home. He entered the Department of 
Physics and Mathematics at Moscow University. As a student, he developed a new method for approximating 
the roots of equations. He graduated from Moscow University in 1841 with a degree in mathematics, and he 
continued his studies, passing his master’s exam in 1843 and completing his master’s thesis in 1846. 
Chebyshev was appointed in 1847 to a position as an assistant at the University of St. Petersburg. He 
wrote and defended a thesis in 1847. He became a professor at St. Petersburg in 1860, a position he held 
until 1882. His book on the theory of congruences written in 1849 was influential in the development of 
number theory. His work on the distribution of prime numbers was seminal. He proved Bertrand’s conjec- 
ture that for every integer n > 3, there is a prime between n and 2n — 2. Chebyshev helped develop ideas that 


were later used to prove the prime number theorem. Chebyshev’s work on the approximation of functions using polynomials is 
used extensively when computers are used to find values of functions. Chebyshev was also interested in mechanics. He studied the 
conversion of rotary motion into rectilinear motion by mechanical coupling. The Chebyshev parallel motion is three linked bars 
approximating rectilinear motion. 
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7.4.8 Chebyshev’s Inequality 


How likely is it that a random variable takes a value far from its expected value? Theorem 8, 
called Chebyshev’s inequality, helps answer this question by providing an upper bound on the 
probability that the value of a random variable differs from the expected value of the random 
variable by more than a specified amount. 


CHEBYSHEV’S INEQUALITY Let X be a random variable on a sample space S with 
probability function p. If r is a positive real number, then 


P(IX(s) — EX)| > r) < VX)”. 


Proof: Let A be the event 
A = {s €S||X(s) — E(X)| > r}. 
What we want to prove is that p(A) < V(X)/r’. Note that 


V(X) = $ X6) - EQDY’p(s) 


ses 


= XX6) - EO p(s) + X X6) - EXP). 
sEA sA 


The second sum in this expression is nonnegative, because each of its summands is nonnegative. 
Also, because for each element s in A, (X(s) — E(X))? > r?°, the first sum in this expression is 
at least -4 p(s). Hence, V(X) > X e4 p(s) = r?p(A). It follows that V(X)/r? > p(A), so 
p(A) < V(X)/r*, completing the proof. < 


Deviation from the Mean when Counting Tails Suppose that X is the random variable that 
counts the number of tails when a fair coin is tossed n times. Note that X is the number of 
successes when n independent Bernoulli trials, each with probability of success 1/2, are per- 
formed. It follows that E(X) = n/2 (by Theorem 2) and V(X) = n/4 (by Example 18). Applying 
Chebyshev’s inequality with r = s/n shows that 


pX(s) —n/2| > Vn) < n/4)/(/ny? = 1/4. 


Consequently, the probability is no more than 1/4 that the number of tails that come up when 
a fair coin is tossed n times deviates from the mean by more than y/n. < 


Chebyshev’s inequality, although applicable to any random variable, often fails to provide 
a practical estimate for the probability that the value of a random variable exceeds its mean by 
a large amount. This is illustrated by Example 20. 


Let X be the random variable whose value is the number appearing when a fair die is rolled. 
We have E(X) = 7/2 (see Example 1) and V(X) = 35/12 (see Example 15). Because the only 
possible values of X are 1, 2, 3, 4, 5, and 6, X cannot take a value more than 5/2 from its mean, 
E(X) = 7/2. Hence, p(|X — 7/2| > r) = 0 ifr > 5/2. By Chebyshev’s inequality we know that 
p(X — 7/2| =r) < (35/12)/r. 

For example, when r= 3, Chebyshev’s inequality tells us that p(|X —7/2| > 3) < 
(35/12)/9 = 35/108 x 0.324, which is a poor estimate, because p(|X — 7/2| > 3) = 0. < 


518 7/ Discrete Probability 


Exercises 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


. What is the expected number of heads that come up when 


a fair coin is flipped five times? 


. What is the expected number of heads that come up when 


a fair coin is flipped 10 times? 


. What is the expected number of times a 6 appears when 


a fair die is rolled 10 times? 


. A coin is biased so that the probability a head comes up 


when it is flipped is 0.6. What is the expected number of 
heads that come up when it is flipped 10 times? 


. What is the expected sum of the numbers that appear on 


two dice, each biased so that a 3 comes up twice as often 
as each other number? 


. What is the expected value when a $1 lottery ticket is 


bought in which the purchaser wins exactly $10 million if 
the ticket contains the six winning numbers chosen from 
the set {1, 2,3, ..., 50} and the purchaser wins nothing 
otherwise? 


. The final exam of a discrete mathematics course con- 


sists of 50 true/false questions, each worth two points, 
and 25 multiple-choice questions, each worth four points. 
The probability that Linda answers a true/false question 
correctly is 0.9, and the probability that she answers a 
multiple-choice question correctly is 0.8. What is her ex- 
pected score on the final? 


. What is the expected sum of the numbers that appear 


when three fair dice are rolled? 


. Suppose that the probability that x is in a list of n distinct 


integers is 2/3 and that it is equally likely that x equals 
any element in the list. Find the average number of com- 
parisons used by the linear search algorithm to find x or 
to determine that it is not in the list. 

Suppose that we flip a fair coin until either it comes up 
tails twice or we have flipped it six times. What is the 
expected number of times we flip the coin? 

Suppose that we roll a fair die until a 6 comes up or we 
have rolled it 10 times. What is the expected number of 
times we roll the die? 

Suppose that we roll a fair die until a 6 comes up. 

a) What is the probability that we roll the die n times? 
b) What is the expected number of times we roll the die? 
Suppose that we roll a pair of fair dice until the sum of 
the numbers on the dice is seven. What is the expected 
number of times we roll the dice? 

Show that the sum of the probabilities of a random vari- 
able with geometric distribution with parameter p, where 
0 <p <1, equals 1. 

Show that if the random variable X has the geometric dis- 
tribution with parameter p, and j is a positive integer, then 
pX >j)=(-pyt. 

Let X and Y be the random variables that count the num- 
ber of heads and the number of tails that come up when 
two fair coins are flipped. Show that X and Y are not in- 
dependent. 


17. 


18. 


19. 


* 20. 


Estimate the expected number of integers with 1000 dig- 
its that need to be selected at random to find a prime, if 
the probability a number with 1000 digits is prime is ap- 
proximately 1/2302. 


Suppose that X and Y are random variables and that X and 
Y are nonnegative for all points in a sample space S. Let Z 
be the random variable defined by Z(s) = max(X(s), Y(s)) 
for all elements s E S. Show that E(Z) < E(X) + E(Y). 


Let X be the number appearing on the first die when two 
fair dice are rolled and let Y be the sum of the numbers 
appearing on the two dice. Show that E(X)E(Y) + E(XY). 


Show that if X,, X, ..., X„ are mutually independent ran- 
dom variables, then E([]'_, X) = IIi; EX). 


The conditional expectation of the random variable X 
given the event A from the sample space S is E(X|A) = 
exis) r- P(X =r A). 


21. 


What is the expected value of the sum of the numbers ap- 
pearing on two fair dice when they are rolled given that 
the sum of these numbers is at least nine. That is, what is 
E(X|A) where X is the sum of the numbers appearing on 
the two dice and A is the event that X > 9? 


The law of total expectation states that if the sample 


space S is the disjoint union of the events S$, S,,...,8 


sA 


and X is a random variable, then E(X) = È E(X|S)P(S). 


22. 
23. 


24. 


25. 


26. 


27. 


28. 


Prove the law of total expectations. 


Use the law of total expectation to find the average weight 
of a breeding elephant seal, given that 12% of the breed- 
ing elephant seals are male and the rest are female, and 
the expected weights of a breeding elephant seal is 4200 
pounds for a male and 1100 pounds for a female. 


Let A be an event. Then 74, the indicator random vari- 
able of A, equals 1 if A occurs and equals 0 otherwise. 
Show that the expectation of the indicator random vari- 
able of A equals the probability of A, that is, EU.) = p(A). 


A run is a maximal sequence of successes in a se- 
quence of Bernoulli trials. For example, in the sequence 
S, S, S, F, S, S, F, F, S, where S represents success and F 
represents failure, there are three runs consisting of three 
successes, two successes, and one success, respectively. 
Let R denote the random variable on the set of sequences 
of n independent Bernoulli trials that counts the num- 
ber of runs in this sequence. Find E(R). [Hint: Show 
that R = 2a I;, where J; = 1 if a run begins at the jth 
Bernoulli trial and I= 0 otherwise. Find E(/,) and then 
find El;), where 1 <j <n.] 


Let X(s) be a random variable, where X(s) is a nonneg- 
ative integer for all s € S, and let A, be the event that 
X(s) > k. Show that E(X) = Joa p(A,). 
What is the variance of the number of heads that come 
up when a fair coin is flipped 10 times? 


What is the variance of the number of times a 6 appears 
when a fair die is rolled 10 times? 


29. 


30. 


31. 


32. 


33. 


* 34, 


35. 


36. 


37. 


38. 


39. 


Let X, be the random variable that equals the number of 
tails minus the number of heads when n fair coins are 
flipped. 

a) What is the expected value of X,,? 

b) What is the variance of X,,? 

Show that if X and Y are independent random variables, 

then V(XY) = EXP V (Y) + EYP VX) + V(X)V(Y). 

Let A(X) = E(|X — E(X)|), the expected value of the ab- 

solute value of the deviation of X, where X is a random 

variable. Prove or disprove that A(X + Y) = A(X) + A(Y) 

for all random variables X and Y. 

Provide an example that shows that the variance of the 

sum of two random variables is not necessarily equal to 

the sum of their variances when the random variables are 
not independent. 

Suppose that X, and X, are independent Bernoulli 

trials each with probability 1/2, and let X,= 

(X, + X,)mod 2. 

a) Show that X,, X,, and X, are pairwise independent, 
but X, and X, + X, are not independent. 

b) Show that V(X, +X, +X3) = V(X,)+ V(X) + 
V(X3). 

c) Explain why a proof by mathematical induction of 
Theorem 7 does not work by considering the random 
variables X,, X,, and X}. 

Prove the general case of Theorem 7. That is, show that 

if X,, X,...,X, are pairwise independent ran- 

dom variables on a sample space S, where n 

is a positive integer, then V(X, +X, ++ +X, = 

V(X) + V(X) + + + V(X,). [Hint: Generalize the proof 

given in Theorem 7 for two random variables. Note that 

a proof using mathematical induction does not work; see 

Exercise 33.] 

Use Chebyshev’s inequality to find an upper bound on the 

probability that the number of tails that come up when a 

fair coin is tossed n times deviates from the mean by more 


than 5 y/n. 


Use Chebyshev’s inequality to find an upper bound on the 

probability that the number of tails that come up when 

a biased coin with probability of heads equal to 0.6 is 

tossed n times deviates from the mean by more than y/n. 

Let X be a random variable on a sample space 

S such that X(s)>0 for all s&s. Show that 

p(X(s) > a) < E(X)/a for every positive real number a. 

This inequality is called Markov’s inequality. 

Suppose that the number of cans of soda pop filled in a 

day at a bottling plant is a random variable with an ex- 

pected value of 10,000 and a variance of 1000. 

a) Use Markov’s inequality (Exercise 37) to obtain an 
upper bound on the probability that the plant will fill 
more than 11,000 cans on a particular day. 

b) Use Chebyshev’s inequality to obtain a lower bound 
on the probability that the plant will fill between 9000 
and 11,000 cans on a particular day. 

Suppose that the number of aluminum cans recycled in 

a day at a recycling center is a random variable with an 

expected value of 50,000 and a variance of 10,000. 


* 40. 


*41. 


* 42. 
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a) Use Markov’s inequality (Exercise 37) to find an up- 
per bound on the probability that the center will recy- 
cle more than 55,000 cans on a particular day. 

b) Use Chebyshev’s inequality to provide a lower bound 
on the probability that the center will recycle 40,000 
to 60,000 cans on a certain day. 

Suppose the probability that x is the ith element in a list 

of n distinct integers is i/[n(n+ 1)]. Find the average 

number of comparisons used by the linear search algo- 
rithm to find x or to determine that it is not in the list. 


In this exercise we derive an estimate of the average-case 

complexity of the variant of the bubble sort algorithm 

that terminates once a pass has been made with no inter- 
changes. Let X be the random variable on the set of per- 

mutations of a set of n distinct integers {a}, dz, ..., a, 

with a, < a, < +++ <a, such that X(P) equals the num- 

ber of comparisons used by the bubble sort to put these 
integers into increasing order. 

a) Show that, under the assumption that the input is 
equally likely to be any of the n! permutations of these 
integers, the average number of comparisons used by 
the bubble sort equals E(X). 

b) Use Example 5 in Section 3.3 to show that E(X) < 
n(n — 1)/2. 

c) Show that the sort makes at least one comparison for 
every inversion of two integers in the input. 

d) Let /(P) be the random variable that equals the num- 
ber of inversions in the permutation P. Show that 
E(X) > E(D). 

e) Let J;, be the random variable with J; ,(P) = 1 if a, 
wfecudes a; in P and Z =0 otherwise. Show that 


IP) = Xe pare! (P). 

f) Show that E(D = >, È Elg 

g) Show that EU; ,) = 1/2. (Hin: e that EU; ,) = 
probability that a, precedes a; in a permutation P. 
Then show it is equally likely for a, to precede a; as 
it is for a; to precede a, in a permutation. ] 

h) Use parts (f) and (g) to show that EV) = n(n — 1)/4. 

i) Conclude from parts (b), (d), and (h) that the aver- 
age number of comparisons used to sort n integers is 
O(n’). 

In this exercise we find the average-case complexity of 

the quick sort algorithm, described in the preamble to Ex- 

ercise 50 in Section 5.4, assuming a uniform distribution 
on the set of permutations. 

a) LetX be the number of comparisons used by the quick 
sort algorithm to sort a list of n distinct integers. Show 
that the average number of comparisons used by the 
quick sort algorithm is E(X) (where the sample space 
is the set of all n! permutations of n integers). 

b) Let J,, denote the random variable that equals 1 if the 
jth smallest element and the kth smallest element of 
the initial list are ever compared as the quick sort al- 
gorithm sorts the list and equals 0 otherwise. Show 


n k- 
that X = 2a Dai Lr 
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c) Show that E(X) = $} Ya p(the jth smallest ele- 
ment and the kth smallest element are compared). 

d) Show that p(the jth smallest element and the kth 
smallest element are compared), where k > j, equals 
2/(kK-j+1). 

e) Use parts (c) and (d) to show that E(X)= 
Un + DÈ; 1/) - 2a - 1). 

f) Conclude from part (e) and the fact that Èi 1/jr 
Inn+y, where y = 0.57721... is Euler’s constant, 
that the average number of comparisons used by the 
quick sort algorithm is O(n log n). 

* 43, What is the variance of the number of fixed elements, 
that is, elements left in the same position, of a randomly 
selected permutation of n elements? [Hint: Let X de- 
note the number of fixed points of a random permuta- 
tion. Write X = X, +X, + + + X, where X, = 1 if the 
permutation fixes the ith element and X; = 0 otherwise.] 

The covariance of two random variables X and Y ona sample 
space S, denoted by Cov(X, Y), is defined to be the expected 


Key Terms and Results 


TERMS 


sample space: the set of possible outcomes of an experiment 

event: a subset of the sample space of an experiment 

probability of an event (Laplace’s definition): the number 
of successful outcomes of this event divided by the number 
of possible outcomes 

probability distribution: a function p from the set of all out- 
comes of a sample space S for which 0 < p(x;) < 1 for 
i=1,2,...,.n and Dy, p@;) = 1, where x, ....x, are the 
possible outcomes 

probability of an event E: the sum of the probabilities of the 
outcomes in E 

p(E|F) (conditional probability of E given F): the ratio 
PEN F)/p(F) 

independent events: events E and F such that p(E N F) = 
PEF) 

pairwise independent events: events £}, E, ..., E,, such that 
p(E; N E;) = p(Epp(E;) for all pairs of integers i and j with 
l<j<k<n 

mutually independent events: events E, Fy,...,£, such 

that p(E;, N E, N+ NE; ) = PEPE) + pE) when- 

ever ip J = 1,2,...,m, are integers with | <i, < i < = < 

in < nand m > 2 

random variable: a function that assigns a real number to 
each possible outcome of an experiment 

distribution of a random variable X: the set of pairs (r, p(X = 
r)) for r E X(S) 

uniform distribution: the assignment of equal probabilities 
to the elements of a finite set 

expected value of a random variable: the weighted average 
of a random variable, with values of the random variable 


value of the random variable (X — E(X))(Y — E(Y)). That is, 

Cov(X, Y) = E(X — EXC — E(Y))). 

44. Show that Cov(X, Y) = E(XY) — E(X)E(Y), and use this 
result to conclude that Cov(X, Y) = 0 if X and Y are in- 
dependent random variables. 

45. Show that V(X + Y) = V(X) + V(Y) + 2 Cov(X, Y). 

46. Find Cov(X, Y) if X and Y are the random variables with 
X(@J)) = 2i and Y((ij))= i+ j, where i and j are the 
numbers that appear on the first and second of two dice 
when they are rolled. 

47. When m balls are distributed into n bins uniformly at ran- 
dom, what is the probability that the first bin remains 
empty? 

48. What is the expected number of balls that fall into the first 
bin when m balls are distributed into n bins uniformly at 
random? 

49. What is the expected number of bins that remain empty 
when m balls are distributed into n bins uniformly at 
random? 


weighted by the probability of outcomes, that is, E(X) = 
Lses POX) 

geometric distribution: the distribution of a random variable 
X such that p(X = k) = (1 — p)¥!p fork = 1, 2, ... for some 
real number p with 0 < p < 1 


independent random variables: random variables X and Y 
such that p(X = r; and Y = ry) = p(X =1r,)p(Y = r) forall 
real numbers r; and r, 


variance of a random variable X: the weighted average of the 
square of the difference between the value of X and its ex- 
pected value E(X), with weights given by the probability of 
outcomes, that is, V(X) = Zes (9) — E(X)’ p(s) 


standard deviation of a random variable X: the square root 
of the variance of X, that is, o(X) = y V(X) 


Bernoulli trial: an experiment with two possible outcomes 


probabilistic (or Monte Carlo) algorithm: an algorithm in 
which random choices are made at one or more steps 


probabilistic method: a technique for proving the existence of 
objects in a set with certain properties that proceeds by as- 
signing probabilities to objects and showing that the prob- 
ability that an object has these properties is positive 


RESULTS 


The probability of exactly k successes when n independent 
Bernoulli trials are carried out equals C(n, bpt", where 
p is the probability of success and q = 1 — p is the proba- 
bility of failure. 
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If X and Y are independent random variables, then E(XY) = 
E(X)E(Y). 


Bayes’ theorem: If E and F are events from a sample space S 
such that p(E) + 0 and p(F) # 0, then 


pE | F)p(F) . 
P(E | F)p(F) + pE | F)p(F) 
E(X) = Drexy POX =r; 


linearity of expectations: E(X, +X,+---+X,)=E(X,)+ 
E(X,) + +++ + E(X p) if X,, Xp, ..., X, are random variables. 


Bienaymé’s formula: If X,, X, ..., X, are independent 
random variables, then V(X, +X, + + +X p) = V(X,) + 
VX) + + + V(X,,). 


p(F | E) = 


Chebyshev’s inequality: p(|X(s) — E(X)| > r) < V(X)/r’, 
where X is a random variable with probability function 
p andr is a positive real number. 


Review Questions 


1. a) Define the probability of an event when all outcomes and it is equally likely that this element is any of 


are equally likely. 


the n elements in the list? 


b) What is the probability that you select the six winning 8. a) What is meant by a Bernoulli trial? 
numbers in a lottery if the six different winning num- b) What is the probability of k successes in n indepen- 
bers are selected from the first 50 positive integers? dent Bernoulli trials? 
. a) What conditions should be met by the probabilities c) Whatis the expected value of the number of successes 
assigned to the outcomes from a finite sample space? in n independent Bernoulli trials? 
b) What probabilities should be assigned to the outcome 9. a) What does the linearity of expectations of random 
of heads and the outcome of tails if heads comes up variables mean? 
three times as often as tails? b) How can the linearity of expectations help us find the 
. a) Define the conditional probability of an event E given AP ted number ar Peele Whe Teese ne Come 
aevent F hat when a hatcheck person returns hats at random? 
b) Suppose E is the event that when a die is rolled it 10. a) How can probability be used to solve a decision prob- 
comes up an even number, and F is the event that lem, if a small probability of PIECE 1S acceptable? . 
when a die is rolled it comes up 1, 2, or 3. What is b) How ge quickly determine whether a positive in- 
the probability of F given E? teger is prime, if we are willing to accept a small prob- 
. ability of making an error? 
AEN wa Evente E ana indepencent) 11. State Bayes’ theorem and use it to find p(F | E) if 


b) Suppose F is the event that an even number appears 
when a fair die is rolled, and F is the event that a 5 
or 6 comes up. Are E and F independent? 


. a) What is a random variable? 
b) What are the possible values assigned by the random 
variable X that assigns to a roll of two dice the larger 
number that appears on the two dice? 


. a) Define the expected value of a random variable X. 
b) What is the expected value of the random variable X 
that assigns to a roll of two dice the larger number that 
appears on the two dice? 


. a) Explain how the average-case computational com- 
plexity of an algorithm, with finitely many possible 
input values, can be interpreted as an expected value. 

b) What is the average-case computational complexity 
of the linear search algorithm, if the probability that 
the element for which we search is in the list is 1/3, 


Supplementary Exercises 


1. What is the probability that six consecutive integers will 
be chosen as the winning numbers in a lottery where each 
number chosen is an integer between | and 40 (inclusive)? 


2. In 2012, a player in the Mega Millions lottery picks five 
different integers between | and 56, inclusive, and a sixth 


12. 


13. 


14. 


15. 


p(E | F) = 1/3, p(E | F) = 1/4, and p(F) = 2/3, where 

E and F are events from a sample space S. 

a) What does it mean to say that a random variable has 
a geometric distribution with parameter p? 

b) What is the mean of a geometric distribution with pa- 
rameter p? 

a) What is the variance of a random variable? 

b) What is the variance of a Bernoulli trial with proba- 
bility p of success? 

a) What is the variance of the sum of n independent ran- 
dom variables? 

b) What is the variance of the number of successes when 
n independent Bernoulli trials, each with probability p 
of success, are carried out? 

What does Chebyshev’s inequality tell us about the prob- 

ability that a random variable deviates from its mean by 

more than a specified amount? 


integer between 1 and 46, which may duplicate one of the 
earlier five integers. The player wins the jackpot if the first 
five numbers picked match the first five numbers drawn 
and the sixth number matches the sixth number drawn. 


a) What is the probability that a player wins the jackpot? 


522 7/ Discrete Probability 


b) What is the probability that a player wins $250,000, 
which is the prize for matching the first five numbers, 
but not the sixth number, drawn? 

c) What is the probability that a player wins $150 by 
matching exactly three of the first five numbers and 
the sixth number or by matching four of the first five 
numbers but not the sixth number? 

d) What is the probability that a player wins a prize, if a 
prize is given when the player matches at least three 
of the first five numbers or the last number. 


. In 2012, a player in the Powerball lottery picks five dif- 
ferent integers between 1 and 59, inclusive, and a sixth 
integer between 1 and 39, which may duplicate one of 
the earlier five integers. The player wins the jackpot if 
the first five numbers picked match the first five number 
drawn and the sixth number matches the sixth number 
drawn. 

a) What is the probability that a player wins the jackpot? 

b) What is the probability that a player wins $200,000, 
which is the prize for matching the first five numbers, 
but not the sixth number, drawn? 

c) What is the probability that a player wins $100 by 
matching exactly three of the first five and the sixth 
numbers or four of the first five numbers but not the 
sixth number? 

d) What is the probability that a player wins a prize, if a 
prize is given when the player matches at least three 
of the first five numbers or the last number. 

. What is the probability that a hand of 13 cards contains 

no pairs? 

. What is the probability that a 13-card bridge hand con- 

tains 

a) all 13 hearts? 

b) 13 cards of the same suit? 

c) seven spades and six clubs? 

d) seven cards of one suit and six cards of a second suit? 

e) four diamonds, six hearts, two spades, and one club? 

f) four cards of one suit, six cards of a second suit, two 
cards of a third suit, and one card of the fourth suit? 

. What is the probability that a seven-card poker hand 

contains 

a) four cards of one kind and three cards of a second 
kind? 

b) three cards of one kind and pairs of each of two dif- 
ferent kinds? 

c) pairs of each of three different kinds and a single card 
of a fourth kind? 

d) pairs of each of two different kinds and three cards of 
a third, fourth, and fifth kind? 

e) cards of seven different kinds? 

f) aseven-card flush? 

g) aseven-card straight? 

h) aseven-card straight flush? 


An octahedral die has eight faces that are numbered 1 
through 8. 


7. a) What is the expected value of the number that comes 


up when a fair octahedral die is rolled? 


b) What is the variance of the number that comes up 
when a fair octahedral die is rolled? 


A dodecahedral die has 12 faces that are numbered 1 
through 12. 


8. 


10. 


11. 


12. 


13. 


14. 


* 15, 


16. 


a) What is the expected value of the number that comes 
up when a fair dodecahedral die is rolled? 

b) What is the variance of the number that comes up 
when a fair dodecahedral die is rolled? 


. Suppose that a pair of fair octahedral dice is rolled. 


a) What is the expected value of the sum of the numbers 
that come up? 

b) What is the variance of the sum of the numbers that 
come up? 

Suppose that a pair of fair dodecahedral dice is rolled. 

a) What is the expected value of the sum of the numbers 
that come up? 

b) What is the variance of the sum of the numbers that 
come up? 

Suppose that a fair standard (cubic) die and a fair octahe- 

dral die are rolled together. 

a) What is the expected value of the sum of the numbers 
that come up? 

b) What is the variance of the sum of the numbers that 
come up? 

Suppose that a fair octahedral die and a fair dodecahedral 

die are rolled together. 

a) What is the expected value of the sum of the numbers 
that come up? 

b) What is the variance of the sum of the numbers that 
come up? 

Suppose n people, n > 3, play “odd person out” to decide 

who will buy the next round of refreshments. The n peo- 

ple each flip a fair coin simultaneously. If all the coins 

but one come up the same, the person whose coin comes 

up different buys the refreshments. Otherwise, the people 

flip the coins again and continue until just one coin comes 

up different from all the others. 

a) What is the probability that the odd person out is de- 
cided in just one coin flip? 

b) What is the probability that the odd person out is de- 
cided with the kth flip? 

c) What is the expected number of flips needed to decide 
odd person out with n people? 

Suppose that p and g are primes and n = pq. What is the 

probability that a randomly chosen positive integer less 

than n is not divisible by either p or q? 

Suppose that m and n are positive integers. What is the 

probability that a randomly chosen positive integer less 

than mn is not divisible by either m or n? 


Suppose that £, E>, ...,E, are n events with p(E;) > 0 
for i = 1, 2, ..., n. Show that 


pE NE) NN E,) 
= P(E, )p(E, | E,)p(E3 | E; N E2) 
DE, | E NE, N+: Epa). 


17. 


18. 


19. 


20. 


21. 


22. 


23 


There are three cards in a box. Both sides of one card are 

black, both sides of one card are red, and the third card 

has one black side and one red side. We pick a card at 

random and observe only one side. 

a) If the side is black, what is the probability that the 
other side is also black? 


b) What is the probability that the opposite side is the 
same color as the one we observed? 

What is the probability that when a fair coin is flipped n 

times an equal number of heads and tails appear? 


What is the probability that a randomly selected bit string 
of length 10 is a palindrome? 


What is the probability that a randomly selected bit string 
of length 11 is a palindrome? 


Consider the following game. A person flips a coin re- 

peatedly until a head comes up. This person receives a 

payment of 2” dollars if the first head comes up at the 

nth flip. 

a) Let X be a random variable equal to the amount of 
money the person wins. Show that the expected value 
of X does not exist (that is, it is infinite). Show that 
a rational gambler, that is, someone willing to pay 
to play the game as long as the price to play is not 
more than the expected payoff, should be willing to 
wager any amount of money to play this game. (This 
is known as the St. Petersburg paradox. Why do you 
suppose it is called a paradox?) 


b) Suppose that the person receives 2” dollars if the first 
head comes up on the nth flip where n < 8 and 28 = 
256 dollars if the first head comes up on or after the 
eighth flip. What is the expected value of the amount 
of money the person wins? How much money should 
a person be willing to pay to play this game? 

Suppose that n balls are tossed into b bins so that each 

ball is equally likely to fall into any of the bins and that 

the tosses are independent. 

a) Find the probability that a particular ball lands in a 
specified bin. 

b) What is the expected number of balls that land in a 
particular bin? 

c) What is the expected number of balls tossed until a 
particular bin contains a ball? 

*d) What is the expected number of balls tossed until all 
bins contain a ball? [Hint: Let X, denote the number 
of tosses required to have a ball land in an ith bin once 
i — 1 bins contain a ball. Find E(X;) and use the lin- 
earity of expectations. ] 


. Suppose that A and B are events with probabilities p(A) = 


3/4 and p(B) = 1/3. 

a) What is the largest p(A N B) can be? What is the small- 
est it can be? Give examples to show that both ex- 
tremes for p(A N B) are possible. 

b) What is the largest p(A U B) can be? What is the small- 
est it can be? Give examples to show that both ex- 
tremes for p(A U B) are possible. 
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24. Suppose that A and B are events with probabilities p(A) = 
2/3 and p(B) = 1/2. 
a) Whatis the largest p(A N B) can be? What is the small- 
est it can be? Give examples to show that both ex- 
tremes for p(A N B) are possible. 


b) Whatis the largest p(A U B) can be? What is the small- 
est it can be? Give examples to show that both ex- 
tremes for p(A U B) are possible. 

25. Recall from Definition 5 in Section 7.2 that the 
events E, E, ..., E, are mutually independent if 
P(E; NE, N+ NE; ) = pE PE; ) + pE) whenever 
i j= 1,2,...,m, are integers with 1 <i, <i, <= < 
in <nandm > 2. 

a) Write out the conditions required for three 
events E}, E,, and E, to be mutually independent. 


b) Let E,, E,, and E; be the events that the first flip comes 
up heads, that the second flip comes up tails, and that 
the third flip comes up tails, respectively, when a fair 
coin is flipped three times. Are E}, E,, and E} mutu- 
ally independent? 


c) Let E,, E,, and E, be the events that the first flip 
comes up heads, that the third flip comes up heads, 
and that an even number of heads come up, respec- 
tively, when a fair coin is flipped three times. Are E}, 
E,, and E, pairwise independent? Are they mutually 
independent? 


d) Let £,, E,, and E; be the events that the first flip comes 
up heads, that the third flip comes up heads, and that 
exactly one of the first flip and third flip come up 
heads, respectively, when a fair coin is flipped three 
times. Are E,, E,, and E, pairwise independent? Are 
they mutually independent? 


e) How many conditions must be checked to show that n 
events are mutually independent? 

26. Suppose that A and B are events from a sample 
space § such that p(A) # 0 and p(B) # 0. Show that if 
p(B | A) < p(B), then p(A | B) < p(A). 

In Exercise 27 we consider the two children problem, intro- 
duced in 1959 by Martin Garnder is his Mathematical Games 
column in Scientific American. A version of the puzzle asks: 
“We meet Mr. Smith as he is walking down the street with a 
young child whom he introduces as his son. He also tells us 
that he has two children. What is the probability that his other 
child is a son?” We will show that this puzzle is ambiguous, 
leading to a paradox, by showing that there are two reasonable 
answers to this problem and we will describe how to make the 
puzzle unambiguous. 


*27. a) Solve this puzzle in two different ways. First, an- 
swer the problem by considering the probability of 
the gender of the second child. Then, determine the 
probability differently, by considering the four differ- 
ent possibilities for a family of two children. 


b) Show that the answer to the puzzle becomes unam- 
biguous if we also know that Mr. Smith chose his 
walking companion at random from his two children. 
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28. 


29. 


30. 


31. 


32. 


33. 


c) Another variation of this puzzle asks “When we meet 
Mr. Smith, he tells us that he has two children and at 
least one is a son. What is the probability that his other 
child is a son?” Solve this variation of the puzzle, ex- 
plaining why it is unambiguous. 

In 2010, the puzzle designer Gary Foshee posed this 
problem: “Mr. Smith has two children, one of whom is 
a son born on a Tuesday. What is the probability that Mr. 
Smith has two sons?” Show that there are two different 
answers to this puzzle, depending on whether Mr. Smith 
specifically mentioned his son because he was born on 
a Tuesday or whether he randomly chose a child and re- 
ported its gender and birth day of the week. [Hint: For 
the first possibility, enumerate all the equally likely pos- 
sibilities for the gender and birth day of the week of the 
other child. To do, this consider first the cases where the 
older child is a boy born on a Tuesday and then the case 
where the older child is not a boy born on a Tuesday. ] 


Let X be a random variable on a sample space S. Show 
that V(aX + b) = a° V(X) whenever a and b are real num- 
bers. 


Use Chebyshev’s inequality to show that the probability 
that more than 10 people get the correct hat back when 
a hatcheck person returns hats at random does not ex- 
ceed 1/100 no matter how many people check their hats. 
[Hint: Use Example 6 and Exercise 43 in Section 7.4.] 


Suppose that at least one of the events E; J= Zs M 
is guaranteed to occur and no more than two can oc- 
cur. Show that if p(E;) = q for j = 1, 2, ...,m and p(E; N 
E) =rfor1 <j< k< m, thenq > 1/mandr < 2/m. 
Show that if m is a positive integer, then the probability 
that the mth success occurs on the (m + n)th trial when 
independent Bernoulli trials, each with probability p of 
success, are run, is (oe )q"p". 

There are n different types of collectible cards you can 
get as prizes when you buy a particular product. Suppose 
that every time you buy this product it is equally likely 
that you get any type of these cards. Let X be the ran- 
dom variable equal to the number of products that need 
to be purchased to obtain at least one of each type of card 
and let X, be the random variable equal to the number of 
additional products that must be purchased after j differ- 
ent cards have been collected until a new card is obtained 
forj=0,1,...,n— 1. 


a) Show that X = X» 2o X; 

b) Show that after j distinct types of cards have been ob- 
tained, the card obtained with the next purchase will 
be a card of a new type with probability (n — j)/n. 

c) Show that X, has a geometric distribution with param- 
eter (n —j)/n. 

d) Use parts (a) and (c) to show that E(X) = n Xi 1/j. 

e) Use the approximation Lai 1/jxlnn+y, where 
y = 0.57721 ... is Euler’s constant, to find the ex- 
pected number of products that you need to buy to get 
one card of each type if there are 50 different types of 
cards. 


34. 


35. 


*36. 


The maximum satisfiability problem asks for an as- 
signment of truth values to the variables in a compound 
proposition in conjunctive normal form (which expresses 
a compound proposition as the conjunction of clauses 
where each clause is the disjunction of two or more vari- 
ables or their negations) that makes as many of these 
clauses true as possible. For example, three but not four 
of the clauses in 


(DV gq) A(PV 7g) A (7p Vr) apy r) 


can be made true by an assignment of truth values to p, q, 
and r. We will show that probabilistic methods can pro- 
vide a lower bound for the number of clauses that can be 
made true by an assignment of truth values to the vari- 
ables. 


a) Suppose that there are n variables in a compound 
proposition in conjunctive normal form. If we pick 
a truth value for each variable randomly by flipping 
a coin and assigning true to the variable if the coin 
comes up heads and false if it comes up tails, what is 
the probability of each possible assignment of truth 
values to the n variables? 


b) Assuming that each clause is the disjunction of ex- 
actly two distinct variables or their negations, what is 
the probability that a given clause is true, given the 
random assignment of truth values from part (a)? 


c) Suppose that there are D clauses in the compound 
proposition. What is the expected number of these 
clauses that are true, given the random assignment of 
truth values of the variables? 


d) Use part (c) to show that for every compound proposi- 
tion in conjunctive normal form there is an assignment 
of truth values to the variables that makes at least 3/4 
of the clauses true. 


What is the probability that each player has a hand con- 
taining an ace when the 52 cards of a standard deck are 
dealt to four players? 


The following method can be used to generate a ran- 
dom permutation of a sequence of n terms. First, in- 
terchange the nth term and the r(n)th term where r(n) 
is a randomly selected integer with 1 < r(n) < n. 
Next, interchange the (n—1)st term of the result- 
ing sequence with its r(n—1)st term where r(n-— 1) 
is a randomly selected integer with 1 < r(n-— 1) < 
n—1. Continue this process until 7 =n, where at 
the jth step you interchange the (n —j + 1)st term of 
the resulting sequence with its r(n—j+1)st term, 
where r(n —j + 1) is a randomly selected integer with 
l<ra-j+1)<n-j+1. Show that when this 
method is followed, each of the n! different permuta- 
tions of the terms of the sequence is equally likely to be 
generated. [Hint: Use mathematical induction, assuming 
that the probability that each of the permutations of n — 1 
terms produced by this procedure for a sequence of n — 1 
terms is 1/(n — 1)!.] 


Computer Projects 


Write programs with these input and output. 


. Given a real number p with O < p < 1, generate random 
numbers taken from a Bernoulli distribution with proba- 


bility p. 
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counting the number of comparisons used. Determine the 
average number of comparisons used over all m permu- 
tations. 


. Given a positive integer n, generate a random permutation 7. Given a positive integer m, simulate the collection of 
of the set {1, 2, 3, ..., n}. (See Exercise 36 in the Supple- cards that come with the purchase of products to find the 
mentary Exercises.) number of products that must be purchased to obtain a 

. Given positive integers m and n, generate m random per- full set of m different collector cards. (See Supplemen- 
mutations of the first n positive integers. Find the num- tary Exercise 33.) 
ber of inversions in each Peta and determine the 8. Given positive integers m and n, simulate the placement 
average number of these inversions. of n keys, where a record with key k is placed at location 

. Given a positive integer n, simulate n repeated flips of a h(k) = k mod m and determine whether there is at least 
biased coin with probability p of heads and determine the one collision. 
ee oP nese ee SoS We DADE Mie Sumulave 9. Given a positive integer n, find the probability of select- 

i oo ing the six integers from the set {1, 2, ..., n} that were 

. Given positive integers n and m, generate m random per- . : 

i eee mechanically selected in a lottery. 
mutations of the first n positive integers. Sort each per- 
mutation using the insertion sort, counting the number 10. Simulate repeated trials of the Monty Hall Three-Door 
of comparisons used. Determine the average number of problem (Example 10 in Section 7.1) to calculate the 
comparisons used over all m permutations. probability of winning with each strategy. 

. Given positive integers n and m, generate m random 11. Given a list of words and the empirical probabilities they 


permutations of the first n positive integers. Sort each 
permutation using the version of the bubble sort that ter- 
minates when a pass has been made with no interchanges, 


occur in spam e-mails and in e-mails that are not spam, 
determine the probability that a new e-mail message is 
spam. 


Computations and Explorations 
Use a computational program or programs you have written to do these exercises. 


1. Find the probabilities of each type of hand in five-card 5. Generate a list of 100 randomly selected permutations of 


poker and rank the types of hands by their probability. 


. Find some conditions such that the expected value of buy- 
ing a $1 lottery ticket in the New Jersey Pick-6 lottery has 


the set of the first 100 positive integers. (See Exercise 36 
in the Supplementary Exercises.) 


z 6. Given a collection of e-mail messages, each determined to 
an expected value of more than $1. To win you have to se- be spam or not to be spam, develop a Bayesian filter based 
lect the six number s drawn, wher c or der does not matter, on the appearance of particular words in these messages. 
from the positive integers 1 to 49, inclusive. The winnings 
are split evenly among holders of winning tickets. Be sure 7. Simulate the odd-person-out procedure (described in Ex- 
to consider the total size of the pot going into the drawing ercise 13 of the Supplementary Exercises) for n peo- 
and the number of people buying tickets. ple with 3 <n< 10. Run a large number of trials for 

. Estimate the probability that two integers selected at ran- each value of n and use the results to estimate the ex- 
dom are relatively prime by testing a large number of ran- pected number of flips needed to find the odd person out. 
domly selected pairs of integers. Look up the theorem that Does your result agree with that found m, Exercise 29 in 
gives this probability and compare your results with the Section 7.2? Vary the problem by supposing that exactly 
correct probability. one person has a biased coin with probability of heads 

. Determine the number of people needed to ensure that the Be 
probability at least two of them have the same day of the 8. Given a positive integer n, simulate a hatcheck person ran- 


year as their birthday is at least 70%, at least 80%, at least 
90%, at least 95%, at least 98%, and at least 99%. 


domly giving hats back to people. Determine the number 
of people who get the correct hat back. 
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Writing Projects 


Respond to these with essays using outside sources. 


. Describe the origins of probability theory and the first uses 
of this theory, including those by Cardano, Pascal, and 
Laplace. 

. Describe the different bets you can make when you 
play roulette. Find the probability of each of these bets 
in the American version where the wheel contains the 
numbers 0 and 00. Which is the best bet and which is the 
worst for you? 

. Discuss the probability of winning when you play the game 
of blackjack versus a casino. Is there a winning strategy for 
the person playing against the house? 

. Investigate the game of craps and discuss the probability 
that the shooter wins and how close to a fair game it is. 


. Discuss issues involved in developing successful spam fil- 


ters and the current situation in the war between spammers 
and people trying to filter spam out. 


. Discuss the history and solution of what is known as 


the Newton—Pepys problem, which asks which is most 
likely: rolling at least one six when six dice are rolled, 
rolling at least two sixes when 12 dice are rolled, or rolling 
at least three sixes when 18 dice are rolled. 


. Explain how Erdős and Rényi first used the probabilis- 


tic method and describe some other applications of this 
method. 


. Discuss the different types of probabilistic algorithms and 


describe some examples of each type. 


8.1 Applications of 
Recurrence 
Relations 


8.2 Solving Linear 
Recurrence 
Relations 


8.3 Divide-and- 
Conquer 
Algorithms and 
Recurrence 
Relations 


8.4 Generating 
Functions 


8.5 Inclusion- 
Exclusion 


8.6 Applications of 
Inclusion— 
Exclusion 


Advanced Counting Techniques 


M» counting problems cannot be solved easily using the methods discussed in Chap- 
ter 6. One such problem is: How many bit strings of length n do not contain two consec- 
utive zeros? To solve this problem, let a, be the number of such strings of length n. An argument 
can be given that shows that the sequence {a,,} satisfies the recurrence relation a, ,, = a, + 4p] 
and the initial conditions a, = 2 and a, = 3. This recurrence relation and the initial conditions 
determine the sequence {a,,}. Moreover, an explicit formula can be found for a,, from the equa- 
tion relating the terms of the sequence. As we will see, a similar technique can be used to solve 
many different types of counting problems. 

We will discuss two ways that recurrence relations play important roles in the study of 
algorithms. First, we will introduce an important algorithmic paradigm known as dynamic 
programming. Algorithms that follow this paradigm break down a problem into overlapping 
subproblems. The solution to the problem is then found from the solutions to the subproblems 
through the use of a recurrence relation. Second, we will study another important algorithmic 
paradigm, divide-and-conquer. Algorithms that follow this paradigm can be used to solve a prob- 
lem by recursively breaking it into a fixed number of nonoverlapping subproblems until these 
problems can be solved directly. The complexity of such algorithms can be analyzed using a spe- 
cial type of recurrence relation. In this chapter we will discuss a variety of divide-and-conquer 
algorithms and analyze their complexity using recurrence relations. 

We will also see that many counting problems can be solved using formal power series, 
called generating functions, where the coefficients of powers of x represent terms of the sequence 
we are interested in. Besides solving counting problems, we will also be able to use generating 
functions to solve recurrence relations and to prove combinatorial identities. 

Many other kinds of counting problems cannot be solved using the techniques discussed in 
Chapter 6, such as: How many ways are there to assign seven jobs to three employees so that 
each employee is assigned at least one job? How many primes are there less than 1000? Both 
of these problems can be solved by counting the number of elements in the union of sets. We 
will develop a technique, called the principle of inclusion—exclusion, that counts the number of 
elements in a union of sets, and we will show how this principle can be used to solve counting 
problems. 

The techniques studied in this chapter, together with the basic techniques of Chapter 6, can 
be used to solve many counting problems. 


Applications of Recurrence Relations 


8.1.1 Introduction 


Recall from Chapter 2 that a recursive definition of a sequence specifies one or more initial terms 
and a rule for determining subsequent terms from those that precede them. Also, recall that a 
rule of the latter sort (whether or not it is part of a recursive definition) is called a recurrence 
relation and that a sequence is called a solution of a recurrence relation if its terms satisfy the 
recurrence relation. 

In this section we will show that such relations can be used to study and to solve counting 
problems. For example, suppose that the number of bacteria in a colony doubles every hour. If 
a colony begins with five bacteria, how many will be present in n hours? To solve this problem, 
let a, be the number of bacteria at the end of n hours. Because the number of bacteria doubles 
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Reproducing pairs 
(at least two months old) 


every hour, the relationship a,, = 2a,__, holds whenever n is a positive integer. This recurrence 
relation, together with the initial condition ag = 5, uniquely determines a, for all nonnegative 
integers n. We can find a formula for a, using the iterative approach followed in Chapter 2, 
namely that a„ = 5 - 2” for all nonnegative integers n. 

Some of the counting problems that cannot be solved using the techniques discussed in 
Chapter 6 can be solved by finding recurrence relations involving the terms of a sequence, as 
was done in the problem involving bacteria. In this section we will study a variety of counting 
problems that can be modeled using recurrence relations. In Chapter 2 we developed methods 
for solving certain recurrence relation. In Section 8.2 we will study methods for finding explicit 
formulae for the terms of sequences that satisfy certain types of recurrence relations. 

We conclude this section by introducing the algorithmic paradigm of dynamic program- 
ming. After explaining how this paradigm works, we will illustrate its use with an example. 


8.1.2 Modeling With Recurrence Relations 


We can use recurrence relations to model a wide variety of problems, such as finding compound 
interest (see Example 11 in Section 2.4), counting rabbits on an island, determining the number 
of moves in the Tower of Hanoi puzzle, and counting bit strings with certain properties. 


Example 1 shows how the population of rabbits on an island can be modeled using a recur- 
rence relation. 


Rabbits and the Fibonacci Numbers Consider this problem, which was originally posed by 
Leonardo Pisano, also known as Fibonacci, in the thirteenth century in his book Liber abaci. A 
young pair of rabbits (one of each sex) is placed on an island. A pair of rabbits does not breed 
until they are 2 months old. After they are 2 months old, each pair of rabbits produces another 
pair each month, as shown in Figure 1. Find a recurrence relation for the number of pairs of 
rabbits on the island after n months, assuming that no rabbits ever die. 


Young pairs Reproducing | Young | Total 
(less than two months old) Month pairs pairs pairs 


aa 1 0 1 1 


aa 2 0 1 1 


aa 


aa 3 1 1 2 


an 


aa aa 4 1 2 3 


aA aA 


Sa ba aa 5 2 3 5 


Sa aA 


aa aa i aa 6 3 5 8 
ana aa 


FIGURE 1 Rabbits on an island. 


The Fibonacci numbers 
appear in many other 
places in nature, 
including the number of 
petals on flowers and the 
number of spirals on 
seedheads. 
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Solution: Denote by f, the number of pairs of rabbits after n months. We will show that f,, 
n = 1,2,3,..., are the terms of the Fibonacci sequence. 

The rabbit population can be modeled using a recurrence relation. At the end of the first 
month, the number of pairs of rabbits on the island is f; = 1. Because this pair does not 
breed during the second month, f, = 1 also. To find the number of pairs after n months, add 
the number on the island the previous month, f,_,, and the number of newborn pairs, which 
equals f,_,, because each newborn pair comes from a pair at least 2 months old. 

Consequently, the sequence {f,} satisfies the recurrence relation 


fa = fa- Pfa 


for n > 3 together with the initial conditions f} = 1 and f, = 1. Because this recurrence relation 
and the initial conditions uniquely determine this sequence, the number of pairs of rabbits on 
the island after n months is given by the nth Fibonacci number. < 


Example 2 involves a famous puzzle. 


The Tower of Hanoi Puzzle A popular puzzle of the late nineteenth century invented by 
the French mathematician Édouard Lucas, called the Tower of Hanoi, consists of three pegs 
mounted on a board together with disks of different sizes. Initially these disks are placed on the 
first peg in order of size, with the largest on the bottom (as shown in Figure 2). The rules of the 
puzzle allow disks to be moved one at a time from one peg to another as long as a disk is never 
placed on top of a smaller disk. The goal of the puzzle is to have all the disks on the second peg 
in order of size, with the largest on the bottom. 

Let H „ denote the number of moves needed to solve the Tower of Hanoi puzzle with n disks. 
Set up a recurrence relation for the sequence {H,,}. 


Solution: Begin with n disks on peg 1. We can transfer the top n — 1 disks, following the rules 
of the puzzle, to peg 3 using H,,_, moves (see Figure 3 for an illustration of the pegs and disks at 
this point). We keep the largest disk fixed during these moves. Then, we use one move to transfer 
the largest disk to the second peg. Finally, we transfer the n — 1 disks on peg 3 to peg 2 using 
H,,_, Moves, placing them on top of the largest disk, which always stays fixed on the bottom 
of peg 2. This shows that we can solve the Tower of Hano puzzle for n disks using 2H,,_, + 1 
moves. 

We now show that we cannot solve the puzzle for n disks using fewer that 2H,,_, + 1 moves. 
Note that when we move the largest disk, we must have already moved the n — 1 smaller disks 
onto a peg other than peg 1. Doing so requires at least H,,_,; moves. Another move is needed to 


Peg 1 Peg 2 Peg 3 


FIGURE 2 The initial position in the Tower of Hanoi. 
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Peg 1 Peg 2 Peg 3 


FIGURE 3 An intermediate position in the Tower of Hanoi. 


transfer the largest disk. Finally, at least H,,_, more moves are needed to put the n — 1 smallest 
disks back on top of the largest disk. Adding the number of moves required gives us the desired 
lower bound. 

We conclude that 


H, = 2H + 1. 


The initial condition is H; = 1, because one disk can be transferred from peg 1 to peg 2, ac- 
cording to the rules of the puzzle, in one move. 
We can use an iterative approach to solve this recurrence relation. Note that 


H, =2H,_,+1 
= 2(2H,_5+1)+1=2°H,,+2+4+1 
= 2°(2H,_3+1)+2+1=29H,_,+27+24+1 


= 2°17, +2772 42% 3 40-4241 
= 214 2 te +241 
= 27-1. 


We have used the recurrence relation repeatedly to express H,, in terms of previous terms of 
the sequence. In the next to last equality, the initial condition H, = 1 has been used. The last 
equality is based on the formula for the sum of the terms of a geometric series, which can be 
found in Theorem 1 in Section 2.4. 

The iterative approach has produced the solution to the recurrence relation H, = 2H,_,; + 1 
with the initial condition H, = 1. This formula can be proved using mathematical induction. 
This is left for the reader as Exercise 1. 

A myth created to accompany the puzzle tells of a tower in Hanoi where monks are trans- 
ferring 64 gold disks from one peg to another, according to the rules of the puzzle. The myth 
says that the world will end when they finish the puzzle. How long after the monks started will 
the world end if the monks take one second to move a disk? 

From the explicit formula, the monks require 


2% — 1 = 18,446,744,073,709, 551,615 
moves to transfer the disks. Making one move per second, it will take them more than 500 


billion years to complete the transfer, so the world should survive a while longer than it already 
has. < 
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Number of bit strings 
of length n with no 
two consecutive Os: 
Any bit string of length n — 1 with 1 
no two consecutive Os Gn 


; Any bit string of length n — 2 with 
End with a 0: . 1 0 an2 
no two consecutive Os Wee 


Total: Ay = an1 + ya 


End with a 1: | 


FIGURE 4 Counting bit strings of length n with no two consecutive Os. 


Remark: Many people have studied variations of the original Tower of Hanoi puzzle discussed 
in Example 2. Some variations use more pegs, some allow disks to be of the same size, and some 
restrict the types of allowable disk moves. One of the oldest and most interesting variations is 
the Reve’s puzzle,* proposed in 1907 by Henry Dudeney in his book The Canterbury Puzzles. 
The Reve’s puzzle involves pilgrims challenged by the Reve to move a stack of cheese wheels 
of varying sizes from the first of four stools to another stool without ever placing a cheese wheel 
on one of smaller diameter. The Reve’s puzzle, expressed in terms of pegs and disks, follows the 
same rules as the Tower of Hanoi puzzle, except that four pegs are used. Similarly, we can gener- 
alize the Tower of Hanoi puzzle where there are p pegs, where p is an integer greater than three. 
You may find it surprising that no one has been able to establish the minimum number of moves 
required to solve the generalization of this puzzle for p pegs. (Note that there have been some 
published claims that this problem has been solved, but these are not accepted by experts.) How- 
ever, in 2014 Thierry Bousch showed that the minimum number of moves required when there 
are four pegs equals the number of moves used by an algorithm invented by Frame and Stewart 
in 1939. (See Exercises 38—45 and [St94] and [Bo14] for more information.) 


Example 3 illustrates how recurrence relations can be used to count bit strings of a specified 
length that have a certain property. 


Find a recurrence relation and give initial conditions for the number of bit strings of length n 
that do not have two consecutive Os. How many such bit strings are there of length five? 


Solution: Leta, denote the number of bit strings of length n that do not have two consecutive Os. 
We assume that n > 3, so that the bit string has at least three bits. Strings of this sort of length n 
can be divided into those that end in 1 and those that end in 0. The bit strings of length n ending 
with 1 that do not have two consecutive Os are precisely the bit strings of length n — 1 with no 
two consecutive Os with a 1 added at the end. Consequently, there are a,_, such bit strings. 

Bit strings of length n ending with a 0 that do not have two consecutive Os must have 
1 as their (n — 1)st bit; otherwise they would end with a pair of Os. Hence, the bit strings 
of length n ending with a 0 that have no two consecutive Os are precisely the bit strings of 
length n — 2 with no two consecutive Os with 10 added at the end. Consequently, there are a,,_, 
such bit strings. 

We conclude, as illustrated in Figure 4, that 


Ay = An} + an-2 


forn > 3. 


* Reve, more commonly spelled reeve, is an archaic word for governor. 
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EXAMPLE 4 


EXAMPLE 5 


The initial conditions are a, = 2, because both bit strings of length one, 0 and 1 do not have 
consecutive Os, and a, = 3, because the valid bit strings of length two are 01, 10, and 11. To 
obtain as, we use the recurrence relation three times to find that 


a3 =d,+a, =34+2=5, 
dg = 43 +a =54+3=8, 
a; = a4 + a = 8 + 5 = 13. < 


Remark: Note that {a,} satisfies the same recurrence relation as the Fibonacci sequence. Be- 
cause a, =f; and a, = f4 it follows that a, = fi45. 


Example 4 shows how a recurrence relation can be used to model the number of codewords 
that are allowable using certain validity checks. 


Codeword Enumeration A computer system considers a string of decimal digits a valid 
codeword if it contains an even number of O digits. For instance, 1230407869 is valid, 
whereas 120987045608 is not valid. Let a, be the number of valid n-digit codewords. Find 
a recurrence relation for a,,. 


Solution: Note that a, = 9 because there are 10 one-digit strings, and only one, namely, the 
string 0, is not valid. A recurrence relation can be derived for this sequence by considering how 
a valid n-digit string can be obtained from strings of n — 1 digits. There are two ways to form a 
valid string with n digits from a string with one fewer digit. 

First, a valid string of n digits can be obtained by appending a valid string of n — 1 dig- 
its with a digit other than 0. This appending can be done in nine ways. Hence, a valid string 
with n digits can be formed in this manner in 9a,,_, ways. 

Second, a valid string of n digits can be obtained by appending a 0 to a string of length 
n — | that is not valid. (This produces a string with an even number of 0 digits because the in- 
valid string of length n — 1 has an odd number of 0 digits.) The number of ways that this can 
be done equals the number of invalid (n — 1)-digit strings. Because there are 10”! strings of 
length n — 1, and a„_; are valid, there are 10"~' — a,,_, valid n-digit strings obtained by append- 
ing an invalid string of length n — 1 with a 0. 

Because all valid strings of length n are produced in one of these two ways, it follows that 
there are 


a, = 9a, + Gor = Gy-1) 
= 8a,_, +10! 


valid strings of length n. < 


Example 5 establishes a recurrence relation that appears in many different contexts. 


Find a recurrence relation for C,,, the number of ways to parenthesize the product of n + 1 num- 


bers, Xp X4 +X) © ++ + Xp to specify the order of multiplication. For example, C} = 5 because 
there are five ways to parenthesize xọ - x, + X3 - x; to determine the order of multiplication: 


(o + x1) + Xp) + X3 (Xp + Œ + X2)) + X3 o +X) + Oy + x3) 


Xo + (x1 + Xp) + X3) Xo * y+ + X3)). 
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Solution: To develop a recurrence relation for C,, we note that however we insert parentheses 
in the product x9 +x, © X2 +++ + Xp, one operator remains outside all parentheses, namely, 
the operator for the final multiplication to be performed. [For example, in (xo - (xı + X2)) + x3, 
it is the final “-”, while in (% : x1) + @2 : x3) it is the second “-”.] This final operator ap- 
pears between two of the n+ 1 numbers, say, x, and x,,,. There are C,C,_,_; ways to in- 
sert parentheses to determine the order of the n + 1 numbers to be multiplied when the final 
operator appears between x, and x,,,;, because there are C, ways to insert parentheses in the 
product xo * x, © =+ x; to determine the order in which these k + 1 numbers are to be multi- 
plied and C,,_,_; ways to insert parentheses in the product Xg} -X,4 © * +X, to determine the 
order in which these n — k numbers are to be multiplied. Because this final operator can appear 
between any two of the n + 1 numbers, it follows that 


ce 99 


Cn = CoCn-1 + Ci Cn-2 ap eee Cn-2Ĉ1 + Cn-1C0 
n-1 


= > CC, 41° 
k=0 


Note that the initial conditions are Cy = 1 and C} = 1. < 


The recurrence relation in Example 5 can be solved using the method of generating func- 
tions, which will be discussed in Section 8.4. It can be shown that C,, = C(2n,n)/(n + 1) (see 
Exercise 43 in Section 8.4) and that C, ~ — (see [GrKnPa94]). The sequence {C,} is the 
sequence of Catalan numbers, named after Eugène Charles Catalan. This sequence appears 
as the solution of many different counting problems besides the one considered here (see the 


chapter on Catalan numbers in [MiRo91] or [RoTe03] for details). 


8.1.3 Algorithms and Recurrence Relations 


Recurrence relations play an important role in many aspects of the study of algorithms and their 
complexity. In Section 8.3, we will show how recurrence relations can be used to analyze the 
complexity of divide-and-conquer algorithms, such as the merge sort algorithm introduced in 
Section 5.4. As we will see in Section 8.3, divide-and-conquer algorithms recursively divide a 
problem into a fixed number of nonoverlapping subproblems until they become simple enough 
to solve directly. We conclude this section by introducing another algorithmic paradigm known 
as dynamic programming, which can be used to solve many optimization problems efficiently. 

An algorithm follows the dynamic programming paradigm when it recursively breaks down 
a problem into simpler overlapping subproblems, and computes the solution using the solutions 
of the subproblems. Generally, recurrence relations are used to find the overall solution from the 
solutions of the subproblems. Dynamic programming has been used to solve important problems 
in such diverse areas as economics, computer vision, speech recognition, artificial intelligence, 
computer graphics, and bioinformatics. In this section we will illustrate the use of dynamic pro- 
gramming by constructing an algorithm for solving a scheduling problem. Before doing so, we 
will relate the amusing origin of the name dynamic programming, which was introduced by the 
mathematician Richard Bellman in the 1950s. Bellman was working at the RAND Corporation 
on projects for the U.S. military, and at that time, the U.S. Secretary of Defense was hostile 
to mathematical research. Bellman decided that to ensure funding, he needed a name not con- 
taining the word mathematics for his method for solving scheduling and planning problems. 
He decided to use the adjective dynamic because, as he said “it’s impossible to use the word 
dynamic in a pejorative sense” and he thought that dynamic programming was “something not 
even a Congressman could object to.” 


534 8/ Advanced Counting Techniques 


AN EXAMPLE OF DYNAMIC PROGRAMMING The problem we use to illustrate dynamic 
programming is related to the problem studied in Example 7 in Section 3.1. In that problem 
our goal was to schedule as many talks as possible in a single lecture hall. These talks have 
preset start and end times; once a talk starts, it continues until it ends; no two talks can proceed 
at the same time; and a talk can begin at the same time another one ends. We developed a 
greedy algorithm that always produces an optimal schedule, as we proved in Example 12 in 
Section 5.1. Now suppose that our goal is not to schedule the most talks possible, but rather to 
have the largest possible combined attendance of the scheduled talks. 

We formalize this problem by supposing that we have n talks, where talk j begins at 
time tis ends at time eis and will be attended by w; students. We want a schedule that maxi- 
mizes the total number of student attendees. That is, we wish to schedule a subset of talks to 
maximize the sum of w; over all scheduled talks. (Note that when a student attends more than 
one talk, this student is counted according to the number of talks attended.) We denote by T(j) 
the maximum number of total attendees for an optimal schedule from the first j talks, so T(n) is 
the maximal number of total attendees for an optimal schedule for all n talks. 

We first sort the talks in order of increasing end time. After doing this, we renumber the 
talks so that e} < e) < = < e,. We say that two talks are compatible if they can be part of the 
same schedule, that is, if the times they are scheduled do not overlap (other than the possibility 
one ends and the other starts at the same time). We define p(j) to be largest integer i, i < j, 
for which e; < s;, if such an integer exists, and p(j) = 0 otherwise. That is, talk p(j) is the talk 
ending latest among talks compatible with talk j that end before talk j ends, if such a talk exists, 
and p(j) = 0 if there are no such talks. 


EXAMPLE 6 Consider seven talks with these start times and end times, as illustrated in Figure 5. 


Talk 1: start 8 A.M., end 10 A.M. Talk 5: start 8:30 A.M., end 2 P.M. 
Talk 2: start 9 A.M., end 11 A.M. Talk 6: start 11 A.M., end 2 P.M. 
Talk 3: start 10:30 A.M., end 12 noon Talk 7: start 1 P.M., end 2 P.M. 


Talk 4: start 9:30 A.M., end 1 P.M. 
Find p(/) for j = 1, 2, ..., 7. 


Solution: We have p(1) = 0 and p(2) = 0, because no talks end before either of the first two 
talks begin. We have p(3) = 1 because talk 3 and talk 1 are compatible, but talk 3 and talk 2 are 
not compatible; p(4) = 0 because talk 4 is not compatible with any of talks 1, 2, and 3; p(5) = 0 


EUGENE CHARLES CATALAN (1814-1894) Eugène Catalan was born in Bruges, then part of France. 
His father became a successful architect in Paris while Eugéne was a boy. Catalan attended a Parisian school 
for design hoping to follow in his father’s footsteps. At 15, he won the job of teaching geometry to his design 
school classmates. After graduating, Catalan attended a school for the fine arts, but because of his mathematical 
aptitude his instructors recommended that he enter the Ecole Polytechnique. He became a student there, but after 
his first year, he was expelled because of his politics. However, he was readmitted, and in 1835, he graduated 
and won a position at the Collège de Châlons sur Marne. 

In 1838, Catalan returned to Paris where he founded a preparatory school with two other mathemati- 
©Paul Fearn/Alamy Stock CİANS, Sturm and Liouville. After teaching there for a short time, he was appointed to a position at the Ecole 
Photo Polytechnique. He received his doctorate from the Ecole Polytechnique in 1841, but his political activity in 
favor of the French Republic hurt his career prospects. In 1846 Catalan held a position at the Collége de Charlemagne; he was 
appointed to the Lycée Saint Louis in 1849. However, when Catalan would not take a required oath of allegiance to the new Emperor 
Louis-Napoleon Bonaparte, he lost his job. For 13 years he held no permanent position. Finally, in 1865 he was appointed to a chair 
of mathematics at the University of Liège, Belgium, a position he held until his 1884 retirement. 

Catalan made many contributions to number theory and to the related subject of continued fractions. He defined what are now 
known as the Catalan numbers when he solved the problem of dissecting a polygon into triangles using non-intersecting diagonals. 
Catalan is also well known for formulating what was known as the Catalan conjecture. This asserted that 8 and 9 are the only 
consecutive powers of integers, a conjecture not solved until 2003. Catalan wrote many textbooks, including several that became 
quite popular and appeared in as many as 12 editions. Perhaps this textbook will have a 12th edition someday! 
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8amM. 9AM. 10am. 11am. 12noon 1 P.M. 2 P.M. 3 P.M. 


FIGURE 5 A schedule of lectures with the values of p(n) shown. 


because talk 5 is not compatible with any of talks 1, 2, 3, and 4; and p(6) = 2 because talk 6 
and talk 2 are compatible, but talk 6 is not compatible with any of talks 3, 4, and 5. Finally, 
p(T) = 4, because talk 7 and talk 4 are compatible, but talk 7 is not compatible with either of 
talks 5 or 6. < 


To develop a dynamic programming algorithm for this problem, we first develop a key 
recurrence relation. To do this, first note that if j < n, there are two possibilities for an optimal 
schedule of the first j talks (recall that we are assuming that the n talks are ordered by increasing 
end time): (i) talk j belongs to the optimal schedule or (ii) it does not. 


Case (i): We know that talks p(j) + 1, ..., j — 1 do not belong to this schedule, for none of these 
other talks are compatible with talk j. Furthermore, the other talks in this optimal schedule must 
comprise an optimal schedule for talks 1, 2, ..., p( j). For if there were a better schedule for talks 


RICHARD BELLMAN (1920-1984) Richard Bellman, born in Brooklyn, where his father was a grocer, 
spent many hours in the museums and libraries of New York as a child. After graduating high school, he stud- 
ied mathematics at Brooklyn College and graduated in 1941. He began postgraduate work at Johns Hopkins 
University, but because of the war, left to teach electronics at the University of Wisconsin. He was able to con- 
tinue his mathematics studies at Wisconsin, and in 1943 he received his masters degree there. Later, Bellman 
entered Princeton University, teaching in a special U.S. Army program. In late 1944, he was drafted into the 
army. He was assigned to the Manhattan Project at Los Alamos where he worked in theoretical physics. After 
the war, he returned to Princeton and received his Ph.D. in 1946. 

@Alfred Eisenstaedt/The After briefly teaching at Princeton, he moved to Stanford University, where he attained tenure. At Stan- 
LIFE Picture ford he pursued his fascination with number theory. However, Bellman decided to focus on mathemati- 
Collection/Getly Images’ CA] questions arising from real-world problems. In 1952, he joined the RAND Corporation, working on 
multistage decision processes, operations research problems, and applications to the social sciences and medicine. He worked on 
many military projects while at RAND. In 1965 he left RAND to become professor of mathematics, electrical and biomedical 
engineering and medicine at the University of Southern California. 

In the 1950s Bellman pioneered the use of dynamic programming, a technique invented earlier, in a wide range of settings. He 
is also known for his work on stochastic control processes, in which he introduced what is now called the Bellman equation. He 
coined the term curse of dimensionality to describe problems caused by the exponential increase in volume associated with adding 
extra dimensions to a space. He wrote an amazing number of books and research papers with many coauthors, including many on 
industrial production and economic systems. His work led to the application of computing techniques in a wide variety of areas 
ranging from the design of guidance systems for space vehicles, to network optimization, and even to pest control. 

Tragically, in 1973 Bellman was diagnosed with a brain tumor. Although it was removed successfully, complications left him 
severely disabled. Fortunately, he managed to continue his research and writing during his remaining ten years of life. Bellman 
received many prizes and awards, including the first Norbert Wiener Prize in Applied Mathematics and the IEEE Gold Medal of 
Honor. He was elected to the National Academy of Sciences. He was held in high regard for his achievements, courage, and admirable 
qualities. Bellman was the father of two children. 
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1, 2, ..., p(j), by adding talk j, we will have a schedule better than the overall optimal schedule. 
Consequently, in case (i), we have T(j) = w, + T(p())). 


Case (ii): When talk j does not belong to an optimal schedule, it follows that an optimal schedule 
from talks 1, 2, ..., jis the same as an optimal schedule from talks 1, 2, ..., j — 1. Consequently, 
in case (ii), we have T(j) = T(j — 1). Combining cases (i) and (ii) leads us to the recurrence 
relation 


TG) = max(w,; + TEQ), TG — D). 


Now that we have developed this recurrence relation, we can construct an efficient algo- 
rithm, Algorithm 1, for computing the maximum total number of attendees. We ensure that the 
algorithm is efficient by storing the value of each T(j) after we compute it. This allows us to 
compute 7(j) only once. If we did not do this, the algorithm would have exponential worst-case 
complexity. The process of storing the values as each is computed is known as memoization 
and is an important technique for making recursive algorithms efficient. 


ALGORITHM 1 Dynamic Programming Algorithm for Scheduling Talks. 


procedure Maximum Attendees (S4, Sy, ..., 8,1 Start times of talks; 
€),€y,-.-,@,: end times of talks; w,, w3, ..., w,,: number of attendees to talks) 
sort talks by end time and relabel so that e} < e, < + < e, 
for j := 1 ton 
if no job 7 with i < j is compatible with job j 
pj) =0 
else p( j) := max{i — i < j and job i is compatible with job j} 
T(0) := 0 
for j:= 1 ton 


T(j) := max(w; + T(p(y)), TG — D) 
return T(n){T(n) is the maximum number of attendees} 


In Algorithm 1 we determine the maximum number of attendees that can be achieved 
by a schedule of talks, but we do not find a schedule that achieves this maximum. To find 
talks we need to schedule, we use the fact that talk j belongs to an optimal solution for the 
first j talks if and only if w; + T(p(j)) 2 TU — 1). We leave it as Exercise 53 to construct an 
algorithm based on this observation that determines which talks should be scheduled to achieve 
the maximum total number of attendees. 

Algorithm 1 is a good example of dynamic programming as the maximum total atten- 
dance is found using the optimal solutions of the overlapping subproblems, each of which 
determines the maximum total attendance of the first j talks for some j with 1 <j<n-1. 
See Exercises 56 and 57 and Supplementary Exercises 14 and 17 for other examples of 
dynamic programming. 


Exercises 
1. Use mathematical induction to verify the formula derived 3. A vending machine dispensing books of stamps accepts 
in Example 2 for the number of moves required to com- only one-dollar coins, $1 bills, and $5 bills. 
plete the Tower of Hanoi puzzle. 
2. a) Find a recurrence relation for the number of permu- a) Find a recurrence relation for the number of ways 
tations of a set with n elements. to deposit n dollars in the vending machine, where 
b) Use this recurrence relation to find the number of per- the order in which the coins and bills are deposited 


mutations of a set with n elements using iteration. matters. 


*6, 


*10. 


11. 


12. 


b) What are the initial conditions? 
c) How many ways are there to deposit $10 for a book 
of stamps? 


. A country uses as currency coins with values of 1 peso, 


2 pesos, 5 pesos, and 10 pesos and bills with values of 
5 pesos, 10 pesos, 20 pesos, 50 pesos, and 100 pesos. 
Find a recurrence relation for the number of ways to pay 
a bill of n pesos if the order in which the coins and bills 
are paid matters. 


. How many ways are there to pay a bill of 17 pesos using 


the currency described in Exercise 4, where the order in 
which coins and bills are paid matters? 


a) Find a recurrence relation for the number of strictly 
increasing sequences of positive integers that have 1 
as their first term and n as their last term, where n is 
a positive integer. That is, sequences a, dp,... 
where a, =1, a,=n, and a<ay,, 
1,2,...,k—-1. 

b) What are the initial conditions? 


c) How many sequences of the type described in (a) are 
there when n is an integer with n > 2? 


>» ks 
for j= 


. a) Find arecurrence relation for the number of bit strings 


of length n that contain a pair of consecutive Os. 

b) What are the initial conditions? 

c) How many bit strings of length seven contain two 
consecutive 0s? 


. a) Find arecurrence relation for the number of bit strings 


of length n that contain three consecutive Os. 

b) What are the initial conditions? 

c) How many bit strings of length seven contain three 
consecutive 0s? 


. a) Find arecurrence relation for the number of bit strings 


of length n that do not contain three consecutive Os. 
b) What are the initial conditions? 
c) How many bit strings of length seven do not contain 
three consecutive 0s? 


a) Find arecurrence relation for the number of bit strings 
of length n that contain the string 01. 

b) What are the initial conditions? 

c) How many bit strings of length seven contain the 
string 01? 

a) Find a recurrence relation for the number of ways to 
climb n stairs if the person climbing the stairs can take 
one stair or two stairs at a time. 

b) What are the initial conditions? 

c) In how many ways can this person climb a flight of 
eight stairs? 

a) Find a recurrence relation for the number of ways to 
climb n stairs if the person climbing the stairs can take 
one, two, or three stairs at a time. 

b) What are the initial conditions? 

c) In how many ways can this person climb a flight of 
eight stairs? 


A string that contains only Os, 1s, and 2s is called a ternary 
string. 


13. 


14. 


15. 


* 16. 


a 1 


** 18, 


19. 


20. 


21. 
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a) Find a recurrence relation for the number of ternary 
strings of length n that do not contain two consecu- 
tive Os. 

b) What are the initial conditions? 

c) How many ternary strings of length six do not contain 


two consecutive Os? 
a) Find a recurrence relation for the number of ternary 


strings of length n that contain two consecutive Os. 
b) What are the initial conditions? 
c) How many ternary strings of length six contain two 


consecutive Os? 
a) Find a recurrence relation for the number of ternary 
strings of length n that do not contain two consecutive 


Os or two consecutive 1s. 
b) What are the initial conditions? 
c) How many ternary strings of length six do not contain 


two consecutive Os or two consecutive 1s? 
a) Find a recurrence relation for the number of ternary 
strings of length n that contain either two consecutive 


Os or two consecutive 1s. 
b) What are the initial conditions? 
c) How many ternary strings of length six contain two 


consecutive Os or two consecutive 1s? 
a) Find a recurrence relation for the number of ternary 
strings of length n that do not contain consecutive 


symbols that are the same. 
b) What are the initial conditions? 
c) How many ternary strings of length six do not contain 


consecutive symbols that are the same? 

a) Find a recurrence relation for the number of ternary 
strings of length n that contain two consecutive sym- 
bols that are the same. 


b) What are the initial conditions? 
c) How many ternary strings of length six contain con- 


secutive symbols that are the same? 
Messages are transmitted over a communications channel 
using two signals. The transmittal of one signal requires 
1 microsecond, and the transmittal of the other signal re- 
quires 2 microseconds. 


a) Find a recurrence relation for the number of differ- 
ent messages consisting of sequences of these two 
signals, where each signal in the message is imme- 
diately followed by the next signal, that can be sent 
in n microseconds. 

b) What are the initial conditions? 

c) How many different messages can be sent in 10 mi- 
croseconds using these two signals? 


A bus driver pays all tolls, using only nickels and dimes, 
by throwing one coin at a time into the mechanical toll 
collector. 


a) Find a recurrence relation for the number of different 
ways the bus driver can pay a toll of n cents (where 


the order in which the coins are used matters). 
b) In how many different ways can the driver pay a toll 


of 45 cents? 


a) Find the recurrence relation satisfied by R,,, where R, 
is the number of regions that a plane is divided into 
by n lines, if no two of the lines are parallel and no 


three of the lines go through the same point. 
b) Find R, using iteration. 
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*22. a) Find the recurrence relation satisfied by R,,, where R, 
is the number of regions into which the surface of a 
sphere is divided by n great circles (which are the in- 
tersections of the sphere and planes passing through 
the center of the sphere), if no three of the great cir- 
cles go through the same point. 

b) Find R, using iteration. 

*23. a) Find the recurrence relation satisfied by S„, where 

S, is the number of regions into which three- 
dimensional space is divided by n planes if every 
three of the planes meet in one point, but no four of 
the planes go through the same point. 

b) Find S, using iteration. 

24. Find a recurrence relation for the number of bit sequences 
of length n with an even number of Os. 

25. How many bit sequences of length seven contain an even 
number of 0s? 

26. a) Find a recurrence relation for the number of ways 
to completely cover a 2 x n checkerboard with 1 x 2 
dominoes. [Hint: Consider separately the coverings 
where the position in the top right corner of the 
checkerboard is covered by a domino positioned hor- 
izontally and where it is covered by a domino posi- 
tioned vertically. ] 

b) What are the initial conditions for the recurrence re- 
lation in part (a)? 

c) How many ways are there to completely cover a 2 Xx 
17 checkerboard with 1 x 2 dominoes? 

27. a) Finda recurrence relation for the number of ways to lay 
out a walkway with slate tiles if the tiles are red, green, 
or gray, so that no two red tiles are adjacent and tiles of 
the same color are considered indistinguishable. 

b) What are the initial conditions for the recurrence re- 
lation in part (a)? 

c) How many ways are there to lay out a path of seven 
tiles as described in part (a)? 

28. Show that the Fibonacci numbers satisfy the recurrence 
relation f, =5f,-4+3f,-5 for n = 5,6,7, ..., together 
with the initial conditions fọ = 0, fi = 1, f) = 1, f3 = 2, 
and f4 = 3. Use this recurrence relation to show that f5, 
is divisible by 5, for n = 1, 2, 3, .... 

*29, Let S(m, n) denote the number of onto functions from 

a set with m elements to a set with n elements. Show 
that S(m, n) satisfies the recurrence relation 


n-1 


S(m, n) = n” — by C(n, k)S(m, k) 
k=1 


whenever m>n and n> 1, with the initial condition 

Sim, 1) = 1. 

30. a) Write out all the ways the product xo + x1 + X2 + X3 © X4 
can be parenthesized to determine the order of multi- 
plication. 

b) Use the recurrence relation developed in Example 5 
to calculate C,, the number of ways to parenthesize 
the product of five numbers so as to determine the or- 
der of multiplication. Verify that you listed the correct 
number of ways in part (a). 


c) Check your result in part (b) by finding C4, using the 
closed formula for C,, mentioned in the solution of 
Example 5. 


31. a) Use the recurrence relation developed in Example 5 
to determine C;, the number of ways to parenthesize 
the product of six numbers so as to determine the or- 
der of multiplication. 


b) Check your result with the closed formula for C; men- 
tioned in the solution of Example 5. 


* 32. In the Tower of Hanoi puzzle, suppose our goal is to 
transfer all n disks from peg 1 to peg 3, but we cannot 
move a disk directly between pegs 1 and 3. Each move 
of a disk must be a move involving peg 2. As usual, we 
cannot place a disk on top of a smaller disk. 

a) Findarecurrence relation for the number of moves re- 
quired to solve the puzzle for n disks with this added 
restriction. 


b) Solve this recurrence relation to find a formula for the 
number of moves required to solve the puzzle for n 
disks. 


c) How many different arrangements are there of the n 
disks on three pegs so that no disk is on top of a 
smaller disk? 


d) Show that every allowable arrangement of the n disks 
occurs in the solution of this variation of the puzzle. 
Exercises 33-37 deal with a variation of the Josephus 
problem described by Graham, Knuth, and Patashnik in 
[GrKnPa94]. This problem is based on an account by the his- 
torian Flavius Josephus, who was part of a band of 41 Jewish 


inks) rebels trapped in a cave by the Romans during the Jewish- 


Roman war of the first century. The rebels preferred suicide to 
capture; they decided to form a circle and to repeatedly count 
off around the circle, killing every third rebel left alive. How- 
ever, Josephus and another rebel did not want to be killed this 
way; they determined the positions where they should stand to 
be the last two rebels remaining alive. The variation we con- 
sider begins with n people, numbered 1 to n, standing around 
a circle. In each stage, every second person still left alive is 
eliminated until only one survives. We denote the number of 
the survivor by J(n). 


33. Determine the value of J(n) for each integer n with 1 < 
n< 16. 


34. Use the values you found in Exercise 33 to conjecture a 
formula for J(n). [Hint: Write n = 2” + k, where m is a 
nonnegative integer and k is a nonnegative integer less 
than 2”.] 


35. Show that J(n) satisfies the recurrence relation J(2n) = 
2J(n)— 1 and J(2n + 1) =2/(n) +1, for n>1, and 
Jd) = 1. 


36. Use mathematical induction to prove the formula you 
conjectured in Exercise 34, making use of the recurrence 
relation from Exercise 35. 


37. Determine J(100), J(1000), and J(10,000) from your for- 
mula for J(n). 

Exercises 38—45 involve the Reve’s puzzle, the variation of 
the Tower of Hanoi puzzle with four pegs and n disks. Before 
presenting these exercises, we describe the Frame—Stewart al- 
gorithm for moving the disks from peg 1 to peg 4 so that no 
disk is ever on top of a smaller one. This algorithm, given 
the number of disks n as input, depends on a choice of an 
integer k with 1 < k < n. When there is only one disk, move 
it from peg 1 to peg 4 and stop. For n > 1, the algorithm pro- 
ceeds recursively, using these three steps. Recursively move 
the stack of the n — k smallest disks from peg 1 to peg 2, us- 
ing all four pegs. Next move the stack of the k largest disks 
from peg 1 to peg 4, using the three-peg algorithm from 
the Tower of Hanoi puzzle without using the peg holding 
the n—k smallest disks. Finally, recursively move the 
smallest n—k disks to peg 4, using all four pegs. Frame 
and Stewart showed that to produce the fewest moves using 
their algorithm, k should be chosen to be the smallest integer 
such that n does not exceed t, = k(k + 1)/2, the kth triangu- 
lar number, that is, t,_; < n < tą. The long-standing conjec- 
ture, known as Frame’s conjecture, that this algorithm uses 
the fewest number of moves required to solve the puzzle, was 
proved by Thierry Bousch in 2014. 


38. Show that the Reve’s puzzle with three disks can be 
solved using five, and no fewer, moves. 


39. Show that the Reve’s puzzle with four disks can be solved 
using nine, and no fewer, moves. 


40. Describe the moves made by the Frame—Stewart al- 
gorithm, with k chosen so that the fewest moves are 
required, for 
a) 5disks. b) 6disks. cc) 7 disks. d) 8 disks. 

*41. Show that if R(n) is the number of moves used by 
the Frame—Stewart algorithm to solve the Reve’s puzzle 
with n disks, where k is chosen to be the smallest in- 
teger with n < k(k + 1)/2, then R(n) satisfies the recur- 
rence relation R(n) = 2R(n — k) + 2* — 1, with R(O) = 0 
and R(1) = 1. 


* 42. Show that if k is as chosen in Exercise 41, then 
R(n) — R(n— 1) = 2%. 


* 43, Show that if k is as chosen in Exercise 41, then 
RQ) = Yi 2 -(, — m2. 


*44. Use Exercise 43 to give an upper bound on the num- 
ber of moves required to solve the Reve’s puzzle for all 
integers n with 1 <n < 25. 


*45. Show that R(n) is O(/n2V”"). 
Let {a,,} be a sequence of real numbers. The backward dif- 
ferences of this sequence are defined recursively as shown 
next. The first difference Va, is 


Va, = a, T Gy}: 
The (k + Ist difference V‘*!a,, is obtained from V*a, by 


k+l, = yk k 
Vera, = V*a, — V*ay_1- 


46. 


47. 
48. 
49. 
* 50. 


51. 


52. 


* 53. 


54. 


55. 


56. 
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Find Va,, for the sequence {a,,}, where 


a) a, =4. b) a, = 2n. 
H a, =n". d) a, = 2". 
Find V7a,, for the sequences in Exercise 46. 
Show that a,_; =a, — Va,- 


Show that a,_5 = a, — 2Va, + V7a,. 


Prove that a,_, can be expressed in terms of a 
Vilpas N üy 


Va, 


n?’ 


Express the recurrence relation a„ = a,_, + a,» in terms 
of a,, Va, and V7a,,. 


n’ n?’ 


Show that any recurrence relation for the sequence {a,} 
can be written in terms of a,, Va,, V7a,, .... The result- 


ing equation involving the sequences and its differences 
is called a difference equation. 


Construct the algorithm described in the text after Algo- 
rithm 1 for determining which talks should be scheduled 
to maximize the total number of attendees and not just 
the maximum total number of attendees determined by 
Algorithm 1. 


Use Algorithm 1 to determine the maximum number of 
total attendees in the talks in Example 6 if w;, the number 
of attendees of talk i, i = 1, 2, ..., 7, is 

a) 20, 10, 50, 30, 15, 25, 40. 

b) 100, 5, 10, 20, 25, 40, 30. 

c) 2,3, 8,5, 4,7, 10. 

d) 10, 8, 7, 25, 20, 30, 5. 

For each part of Exercise 54, use your algorithm from 


Exercise 53 to find the optimal schedule for talks so that 
the total number of attendees is maximized. 


In this exercise we will develop a dynamic program- 
ming algorithm for finding the maximum sum of con- 
secutive terms of a sequence of real numbers. That 
is, given a sequence of real numbers a, a, ...,a 
k 
pi 


n? 
the algorithm computes the maximum sum J, 
where 1 <j<k<n. 


a) Show that if all terms of the sequence are nonnega- 
tive, this problem is solved by taking the sum of all 
terms. Then, give an example where the maximum 
sum of consecutive terms is not the sum of all terms. 


b) Let M(k) be the maximum of the sums of consecutive 
terms of the sequence ending at a,. That is, M(k) = 
MAX) <j<k È a;. Explain why the recurrence relation 


M(k) = max(M(k — 1) + a, ay) holds for k = 2, ..., n. 
c) Use part (b) to develop a dynamic programming al- 
gorithm for solving this problem. 
d) Show each step your algorithm from part (c) uses to 
find the maximum sum of consecutive terms of the 
sequence 2, —3, 4, 1, —2, 3. 


e) Show that the worst-case complexity in terms of the 
number of additions and comparisons of your algo- 
rithm from part (c) is linear. 
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*57, Dynamic programming can be used to develop 


an algorithm for solving the matrix-chain multi- 
plication problem introduced in Section 3.3. This 
is the problem of determining how the product 
AJA, =A, can be computed using the fewest 
integer multiplications, where Aj, A),...,A, are 
m, X My, My X M, ..., M, X M,,,, Matrices, respectively, 
and each matrix has integer entries. Recall that by the 


associative law, the product does not depend on the order 


least number of integer multiplications are used to 
compute A;, where i<j, by splitting the product 
into the product of A, through A, and the prod- 
uct of A,,; through Aj, then the first k terms must 
be parenthesized so that A is computed in the 
optimal way using M(i,k) integer multiplications, 
and A,,,; must be parenthesized so that A,,,; is 
computed in the optimal way using M(k + 1, j) inte- 
ger multiplications. 


in which the matrices are multiplied. c) Explain why part (b) leads to the recurrence 

a) Show that the brute-force method of determining the relation = M(,j) = minjc, (MG k) + M(k + Lj) + 
minimum number of integer multiplications needed to mm, Mil <i<j<j<n. 
solve a matrix-chain multiplication problem has expo- d) Use the recurrence relation in part (c) to construct 
nential worst-case complexity. [Hint: Do this by first an efficient algorithm for determining the order 
showing that the order of multiplication of matrices the n matrices should be multiplied to use the min- 
is specified by parenthesizing the product. Then, use imum number of integer multiplications. Store the 
Example 5 and the result of part (c) of Exercise 43 in partial results M(@,j) as you find them so that your 
Section 8.4.] algorithm will not have exponential complexity. 

b) Denote by Ay the product A; Ani- A,, e) Show that your algorithm from part (d) has O(n?) 


and M(,j) the minimum number of integer mul- 
tiplications required to find A, Show that if the 


worst-case complexity in terms of multiplications of 
integers. 


Solving Linear Recurrence Relations 


Links » 


Definition 1 


EXAMPLE 1 


8.2.1 Introduction 


A wide variety of recurrence relations occur in models. Some of these recurrence relations 
can be solved using iteration or some other ad hoc technique. However, one important class of 
recurrence relations can be explicitly solved in a systematic way. These are recurrence relations 
that express the terms of a sequence as linear combinations of previous terms. 


A linear homogeneous recurrence relation of degree k with constant coefficients is a recur- 
rence relation of the form 


an = C1 ân- F C2Ay_g Ho + Cran- 


where Cj, Co, ..., Cp are real numbers, and c, # 0. 


The recurrence relation in the definition is linear because the right-hand side is a sum of pre- 
vious terms of the sequence each multiplied by a function of n. The recurrence relation is ho- 
mogeneous because no terms occur that are not multiples of the as. The coefficients of the 
terms of the sequence are all constants, rather than functions that depend on n. The degree 
is k because a, is expressed in terms of the previous k terms of the sequence. 

A consequence of the second principle of mathematical induction is that a sequence satis- 
fying the recurrence relation in the definition is uniquely determined by this recurrence relation 
and the k initial conditions 


Ay = Co ay = Cy, Gy = Ce. 


The recurrence relation P, = (1.11)P,,_, is a linear homogeneous recurrence relation of degree 
one. The recurrence relation f, =f; +f,—2 is a linear homogeneous recurrence relation of 


EXAMPLE 2 
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degree two. The recurrence relation a, = a,_5 is a linear homogeneous recurrence relation of 
degree five. < 


To help clarify the definition of linear homogeneous recurrence relations with constant co- 
efficients, we will now provide examples of recurrence relations each lacking one of the defining 
properties. 


The recurrence relation a, = a,_) + ie is not linear. The recurrence relation H, = 2H,_,+1 
is not homogeneous. The recurrence relation B,, = nB,_, does not have constant coefficients. 


Linear homogeneous recurrence relations are studied for two reasons. First, they often occur 
in modeling of problems. Second, they can be systematically solved. 


8.2.2 Solving Linear Homogeneous Recurrence Relations 
with Constant Coefficients 


Recurrence relations may be difficult to solve, but fortunately this is not the case for linear 
homogenous recurrence relations with constant coefficients. We can use two key ideas to find 
all their solutions. First, these recurrence relations have solutions of the form a, = r”, where 
r is a constant. To see this, observe that a, =r" is a solution of the recurrence relation a, = 
C1Ay_1 + CQy_y + + + Cap- if and only if 
P= ag + or ++ er. 

When both sides of this equation are divided by 7”~* (when r Æ 0) and the right-hand side is 
subtracted from the left, we obtain the equation 


r = or — er? —  — Gg rg, = O. 
Consequently, the sequence {a,,} with a, = r” where r Æ 0 is a solution if and only if r is a 
solution of this last equation. We call this the characteristic equation of the recurrence relation. 
The solutions of this equation are called the characteristic roots of the recurrence relation. As 
we will see, these characteristic roots can be used to give an explicit formula for all the solutions 
of the recurrence relation. 

The other key observation is that a linear combination of two solutions of a linear homoge- 
neous recurrence relation is also a solution. To see this, suppose that s,, and ¢,, are both solutions 
of a, = Cay) + CpG,_7 $e + Capg. Then 


Sn = Cisa- + CaSn-2 Tent CkSn-k 
and 
ta = Ciba- + Coln_2 ale Sle Ckfn=k: 
Now suppose that b, and b, are real numbers. Then 
DiS, T Dot, = bi (Cispa T CaSn-2 Pori CkSn-k) T by(Cyty-1 T Caln-2 A Cktn-k) 
= ci (bi Sp- + Dyty-1) + C7 (DS, F boty) abst CeO Spx + Dit,-4): 


This means that b,s,, + byt, is also a solution of the same linear homogeneous recurrence rela- 
tion. 

Using these key observations, we will show how to solve linear homogeneous recurrence 
relations with constant coefficients. 
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THE DEGREE TWO CASE We now turn our attention to linear homogeneous recurrence 
relations of degree two. First, consider the case when there are two distinct characteristic roots. 


THEOREM 1 Let c} and c, be real numbers. Suppose that 7? — c,r — c, = 0 has two distinct roots r; 
and r,. Then the sequence {a,,} is a solution of the recurrence relation a, = c,d,_, + Co@,_9 
if and only if a, = ar} + œr; for n = 0, 1, 2, ..., where æ; and æ, are constants. 


Proof: We must do two things to prove the theorem. First, it must be shown that if r; and r, 
are the roots of the characteristic equation, and @, and œ, are constants, then the sequence {a,,} 
with a„ = ar; + ar; is a solution of the recurrence relation. Second, it must be shown that if 
the sequence {a,,} is a solution, then a„ = a;r} + ar; for some constants a, and az. 

We now show that if a, = œr} + ar), then the sequence {a,} is a solution of 
the recurrence relation. Because r, and r, are roots of r?—cjr—c,=0, it follows 
that a = Cr, +c and i = Cir, + Cp. 

From these equations, we see that 

CjAy—-1 + Cap- = clari! + ari!) + c(a rt? + ari?) 

= ari? (er +c) + ari (cir +c) 
n=2 2 n=2 2 


ar r+ ar, r3 


— n n 
= ar; + Qr 


= aw 


This shows that the sequence {a,,} with a, = a,r} + ar; is a solution of the recurrence relation. 


To show that every solution {a,} of the recurrence relation a, =c)d,_) + C242 
has a, = œr] + ar} for n = 0, 1, 2, ..., for some constants a, and a5, suppose that {a,,} is a 
solution of the recurrence relation, and the initial conditions a, = Cy and a, = C, hold. It will 
be shown that there are constants a, and a, such that the sequence {a,,} with a, = a;r] + ar; 
satisfies these same initial conditions. This requires that 


dg = Cy = & + M%, 


a) = Cy = ary + ar. 


We can solve these two equations for a, and a,. From the first equation it follows that 
a, = Cy — a@,. Inserting this expression into the second equation gives 


Cy = ari + (Co — a&i )ra. 


Ci = æ (ri — r2) + Cora. 
This shows that 


Ci — Cor 


M1 


a= 


and 


Ci — Corr _ Cory =C] 


fyi = Bp) 


@ = Co — a) = Cy — 
r= n 


EXAMPLE 3 


Extra » 
Examples 


EXAMPLE 4 
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where these expressions for a, and a, depend on the fact that r4 # r,. (When r; = rp, this the- 
orem is not true.) Hence, with these values for a, and a, the sequence {a,} with ar] + œr) 
satisfies the two initial conditions. 

We know that {a,} and {a;r} + œr} are both solutions of the recurrence relation 
Gy = C\4,_1 + C242 and both satisfy the initial conditions when n = 0 and n = 1. Because 
there is a unique solution of a linear homogeneous recurrence relation of degree two with two 
initial conditions, it follows that the two solutions are the same, that is, a, = ar) + ar, for 
all nonnegative integers n. We have completed the proof by showing that a solution of the lin- 
ear homogeneous recurrence relation with constant coefficients of degree two must be of the 
form a, = ar} + azr}, where a, and a, are constants. < 


The characteristic roots of a linear homogeneous recurrence relation with constant coef- 
ficients may be complex numbers. Theorem 1 (and also subsequent theorems in this section) 
still applies in this case. Recurrence relations with complex characteristic roots will not be dis- 
cussed in the text. Readers familiar with complex numbers may wish to solve Exercises 38 
and 39. 

Examples 3 and 4 show how to use Theorem 1 to solve recurrence relations. 


What is the solution of the recurrence relation 

ay = an-1 + 24y-2 
with dy = 2 anda, = 7? 
Solution: Theorem 1 can be used to solve this problem. The characteristic equation of the re- 
currence relation is 7? — r — 2 = 0. Its roots are r = 2 and r = —1. Hence, the sequence {a,,} is 
a solution to the recurrence relation if and only if 

á, = a2" + a,(-1)”, 


for some constants @, and @,. From the initial conditions, it follows that 


dg = 2 = & +%, 
a =7=a,:2+a,-(-l). 


Solving these two equations shows that a, = 3 and a, = —1. Hence, the solution to the recur- 
rence relation and initial conditions is the sequence {a,,} with 


a, = 332" =(1)". 4 


Find an explicit formula for the Fibonacci numbers. 


Solution: Recall that the sequence of Fibonacci numbers satisfies the recurrence relation f, = 
fn—-1 +f,-2 and also satisfies the initial conditions fọ = O and f, = 1. The roots of the character- 
istic equation 7? —r—1=Oarer, = (1 + /5)/2 and r, = (1 — /5)/2. Therefore, from The- 
orem | it follows that the Fibonacci numbers are given by 


145\" 1-s5\" 
a } ta( => 


Sn = % 2 , 
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THEOREM 2 


EXAMPLE 5 


for some constants @, and @,. The initial conditions fọ = 0 and f, = 1 can be used to find these 
constants. We have 


Jo =& + & =O, 


14+ 5 1=4/5 
Ea +a | ——— 


fi=a 2 2 


The solution to these simultaneous equations for a, and a, is 


a, =1/V5, a, =-1/¥5. 


Consequently, the Fibonacci numbers are given by 


fa Laas a. fs) 


Theorem 1 does not apply when there is one characteristic root of multiplicity two. If this 


happens, then a,, = nr) is another solution of the recurrence relation when ro is a root of multi- 


plicity two of the characteristic equation. Theorem 2 shows how to handle this case. 


Let c; and c, be real numbers with c, # 0. Suppose that r? — cır — c, = 0 has only one root 
ry. A sequence {a,,} is a solution of the recurrence relation a, = c,d,_, +C)d,_, if and only 
if a, = arg + anro, for n = 0, 1, 2, ..., where a, and œ, are constants. 


The proof of Theorem 2 is left as Exercise 10. Example 5 illustrates the use of this theorem. 


What is the solution of the recurrence relation 
a, = 64,_| = 9a,» 


with initial conditions ay = 1 and a, = 6? 


Solution: The only root of r? — 6r + 9 = 0 is r = 3. Hence, the solution to this recurrence rela- 
tion is 


a, = 0,3" + @n3" 
for some constants @, and a,. Using the initial conditions, it follows that 


a =1l=a, 
ad, =6=a,:3+a,°3. 


Solving these two equations shows that a, = 1 and a, = 1. Consequently, the solution to this 
recurrence relation and the initial conditions is 


a, = 3" + n3", 


THEOREM 3 


EXAMPLE 6 
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THE GENERAL CASE We will now state the general result about the solution of linear ho- 
mogeneous recurrence relations with constant coefficients, where the degree may be greater 
than two, under the assumption that the characteristic equation has distinct roots. The proof of 
this result will be left as Exercise 16. 


Let c1, Co, ..., c, be real numbers. Suppose that the characteristic equation 
r= ar =e 
has k distinct roots r,, r2, ..., rp. Then a sequence {a,,} is a solution of the recurrence relation 


an = Cian} oF C742 ian Ckân-k 
if and only if 
Ga = Gr, Gof, Fo Or, 


forn = 0, 1, 2,..., where a), a, ..., a, are constants. 


We illustrate the use of the theorem with Example 6. 


Find the solution to the recurrence relation 


a, = 6a,_,; — 1la,_, + 6a,_3 


n 


with the initial conditions dj = 2, a, = 5, and a, = 15. 
Solution: The characteristic polynomial of this recurrence relation is 
r —6r +11r—6. 


The characteristic roots are r=1, r=2, and r=3, because r’ -— 6r? +l1lr-6= 
(r — 1)(r — 2)(r — 3). Hence, the solutions to this recurrence relation are of the form 


a, = a, : 1" + œ +2” +o - 3". 


n 


To find the constants @,, a, and a3, use the initial conditions. This gives 


dy = 2 =a, + + 43, 
a; =S=a,+a,:2+4;-3, 
a = 15 =a, +a :4+:9. 


When these three simultaneous equations are solved for a), a, and a3, we find that a, = 1, 
a, = —1, and a; = 2. Hence, the unique solution to this recurrence relation and the given initial 
conditions is the sequence {a,,} with 

a= l= 2? 42-3", < 


n 


We now state the most general result about linear homogeneous recurrence relations with 
constant coefficients, allowing the characteristic equation to have multiple roots. The key point 
is that for each root r of the characteristic equation, the general solution has a summand of the 
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THEOREM 4 


EXAMPLE 7 


EXAMPLE 8 


form P(n)r", where P(n) is a polynomial of degree m — 1, with m the multiplicity of this root. 
We leave the proof of this result as Exercise 51. 


Let c4, Co, ..., c, be real numbers. Suppose that the characteristic equation 

fF —ort—..-c, =0 
has ft distinct roots r,7>,...,7, With multiplicities m,,mp,...,m,, respectively, so 
that m; > 1 for i = 1, 2, ...,t and m; +m, ++» +m, = k. Then a sequence {a,} is a solu- 


tion of the recurrence relation 
an = Cian] ar C74y_2 Sue Cenk 
if and only if 
2 =j 
A, = Cio r ea +a, 10 rh 


My—1>,n 
T (9 + Qn tee gr Q? m,—1” 2 Y 


aP oa d (Cr r a ears eek 
for n = 0, 1, 2, ..., where a; ; are constants for 1 <i < t and O < j < m; — 1. 
Example 7 illustrates how Theorem 4 is used to find the general form of a solution of a 


linear homogeneous recurrence relation when the characteristic equation has several repeated 
roots. 


Suppose that the roots of the characteristic equation of a linear homogeneous recurrence relation 
are 2, 2, 2, 5, 5, and 9 (that is, there are three roots, the root 2 with multiplicity three, the root 
5 with multiplicity two, and the root 9 with multiplicity one). What is the form of the general 
solution? 


Solution: By Theorem 4, the general form of the solution is 


(aio + an+ Gian )2" + (Ao. + @,1N)5" + 3,9". 4 


We now illustrate the use of Theorem 4 to solve a linear homogeneous recurrence relation 
with constant coefficients when the characteristic equation has a root of multiplicity three. 


Find the solution to the recurrence relation 
a, = ae ~ 34,9 -= an-3 
with initial conditions ag = 1, a, = —2, and a, = —1. 
Solution: The characteristic equation of this recurrence relation is 
3 2 — 
r+3r°+3r+1=0. 


Because r° + 37? +3r + 1 = (r + 1)’, there is a single root r = —1 of multiplicity three of the 
characteristic equation. By Theorem 4 the solutions of this recurrence relation are of the form 


d, = @ o(-1)" + a yn(-1)" + ay yn°(-1)". 


EXAMPLE 9 


THEOREM 5 
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To find the constants a, 9, a, ;, and q; 5, use the initial conditions. This gives 


do = 1 = Q1 
ay = -2 = O19 — 1,1 — a9, 
-1 = &io + 2a; 1 + 4a, >. 


a 
S 
II 


The simultaneous solution of these three equations is œjọ = 1, a; =3, and œ = —2. 
Hence, the unique solution to this recurrence relation and the given initial conditions is the 
sequence {a,,} with 


a, = (1+3n—2n’\(-1)". 4 


8.2.3 Linear Nonhomogeneous Recurrence Relations 
with Constant Coefficients 


We have seen how to solve linear homogeneous recurrence relations with constant coefficients. 
Is there a relatively simple technique for solving a linear, but not homogeneous, recurrence 
relation with constant coefficients, such as a, = 3a,_; + 2n? We will see that the answer is yes 
for certain families of such recurrence relations. 

The recurrence relation a, = 3a,_; + 2n is an example of a linear nonhomogeneous re- 
currence relation with constant coefficients, that is, a recurrence relation of the form 


An = C{An_y + Cody_g + + Cza,- + FC), 


where c4, Cy, ..., C are real numbers and F(n) is a function not identically zero depending only 
on n. The recurrence relation 


an = Cyay_-} F C2đn-2 asea Ckân-k 


is called the associated homogeneous recurrence relation. It plays an important role in the 
solution of the nonhomogeneous recurrence relation. 


Each of the recurrence relations a, = d,_; +2", a, = 4,_; +4,2 +N? +N +1, a, = 3a,_; + 
n3”, and a, = d,_, + 4,2 +.4,_3 +n! is a linear nonhomogeneous recurrence relation with 
constant coefficients. The associated linear homogeneous recurrence relations are a, = d,_1, 
Ay = Ay_-1 + Aya, 4, = 3a,_,, and a, = a,_; +a,_7 + a,_3, respectively. < 


The key fact about linear nonhomogeneous recurrence relations with constant coefficients 
is that every solution is the sum of a particular solution and a solution of the associated linear 
homogeneous recurrence relation, as Theorem 5 shows. 


If {a} is a particular solution of the nonhomogeneous linear recurrence relation with con- 
stant coefficients 


an = Cân- oP Co On -2 sta ccd Cran_k oF F(n), 


then every solution is of the form {a” + a}, where {a} is a solution of the associated 
homogeneous recurrence relation 


an = Cian} oF C2 n-2 fe ae Crank: 
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> 


Proof: Because {a”} is a particular solution of the nonhomogeneous recurrence relation, we 
know that 
gs ca”, + ca”, ++ aP, + F(n). 


n— | 


Now suppose that {b,,} is a second solution of the nonhomogeneous recurrence relation, so that 
b, = cib, + Cyd, 9 +--+ CD n_K + F(n). 
Subtracting the first of these two equations from the second shows that 


(p) 


ney, eara Cba = a” ). 


= (p) 
b,- aP = cı (b, 1 — a) t+ cCa(b, 2 —a A 


It follows that {b, —a?} is a solution of the associated homogeneous linear recurrence, 
say, {a}. Consequently, b, = aP + a“ for all n. < 


By Theorem 5, we see that the key to solving nonhomogeneous recurrence relations with 
constant coefficients is finding a particular solution. Then every solution is a sum of this solution 
and a solution of the associated homogeneous recurrence relation. Although there is no general 
method for finding such a solution that works for every function F(n), there are techniques that 
work for certain types of functions F(n), such as polynomials and powers of constants. This is 
illustrated in Examples 10 and 11. 


Find all solutions of the recurrence relation a„ = 3a,_; + 2n. What is the solution with a, = 3? 


Solution: To solve this linear nonhomogeneous recurrence relation with constant coefficients, 
we need to solve its associated linear homogeneous equation and to find a particular solution 
for the given nonhomogeneous equation. The associated linear homogeneous equation is a, = 
3a,- Its solutions are a”) = a3”, where a is a constant. 

We now find a particular solution. Because F(n) = 2n is a polynomial in n of degree one, a 
reasonable trial solution is a linear function in n, say, p, = cn + d, where c and d are constants. 
To determine whether there are any solutions of this form, suppose that p, = cn +d is such 
a solution. Then the equation a, = 3a,_; + 2n becomes cn +d = 3(c(n — 1) + d) + 2n. Sim- 
plifying and combining like terms gives (2 + 2c)n + (2d — 3c) = 0. It follows that cn + d is a 
solution if and only if 2 + 2c = 0 and 2d — 3c = 0. This shows that cn + d is a solution if and 
only if c = —1 and d = —3/2. Consequently, a” =-n-3 /2 is a particular solution. 

By Theorem 5 all solutions are of the form 


= hy 3 
a= dp + a) =n 5 40-3" 


where a is a constant. 

To find the solution with a, = 3, let n = 1 in the formula we obtained for the general so- 
lution. We find that 3 = —1 — 3/2 + 3a, which implies that a = 11/6. The solution we seek is 
a, = -n — 3/2 + (11/6)3”. < 


Find all solutions of the recurrence relation 


a, = 5da,_; — 64,9 +7". 


n 


Solution: This is a linear nonhomogeneous recurrence relation. The solutions of its associated 
homogeneous recurrence relation 


a, = Sa,- E 6a, 


THEOREM 6 


EXAMPLE 12 
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are a” = a, : 3" + a - 2", where a, and a, are constants. Because F(n) = 7”, a reasonable trial 


solution is a” = C-7", where C is a constant. Substituting the terms of this sequence into 


the recurrence relation implies that C - 7” = 5C -7"-! — 6C - 7"? + 7”. Factoring out 7”~”, this 
equation becomes 49C = 35C — 6C + 49, which implies that 20C = 49, or that C = 49/20. 


Hence, a” = (49 /20)7" is a particular solution. By Theorem 5, all solutions are of the form 


a, = & 3" + œ +2" + (49/20)7". < 


n 


In Examples 10 and 11, we made an educated guess that there are solutions of a particular 
form. In both cases we were able to find particular solutions. This was not an accident. Whenever 
F(n) is the product of a polynomial in n and the nth power of a constant, we know exactly what 
form a particular solution has, as stated in Theorem 6. We leave the proof of Theorem 6 as 
Exercise 52. 


Suppose that {a,,} satisfies the linear nonhomogeneous recurrence relation 
An = CyAq_1 + CoAn_n +0 + CA, + F(n), 

where c}, Cz, ..., Cp are real numbers, and 
F(n) = (bn + b,n! + + bin + bo)s”, 


where bp, b,, ..., b, and s are real numbers. When s is not a root of the characteristic equation 
of the associated linear homogeneous recurrence relation, there is a particular solution of the 
form 


(pn! + p,_yn' | +- + pnt po)". 


When s is a root of this characteristic equation and its multiplicity is m, there is a particular 
solution of the form 


n™(p,n' +p, in! +++ + pin + pos”. 


Note that in the case when s is a root of multiplicity m of the characteristic equation of the 
associated linear homogeneous recurrence relation, the factor n” ensures that the proposed par- 
ticular solution will not already be a solution of the associated linear homogeneous recurrence 
relation. We next provide Example 12 to illustrate the form of a particular solution provided by 
Theorem 6. 


What form does a particular solution of the linear nonhomogeneous recurrence rela- 
tion a, = 6a,_, — 9a,_. + F(n) have when F(n) = 3”, F(n) = 3", F(n) = n°2", and F(n) = 
(n? + 1)3"? 


Solution: The associated linear homogeneous recurrence relation is a, = 6a,_, — 9a,_. Its 
characteristic equation, r? —6r+9=(r—3) =0, has a single root, 3, of multiplicity two. To 
apply Theorem 6, with F'(n) of the form P(n)s”, where P(n) is a polynomial and s is a constant, 
we need to ask whether s is a root of this characteristic equation. 

Because s = 3 is a root with multiplicity m = 2 but s = 2 is not a root, Theorem 6 tells us 
that a particular solution has the form pọn?3” if F(n) = 3”, the form n?(p,n + po)3” if F(n) = 
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n3", the form (pan? + pn + py)2” if F(n) = n72", and the form n? (pan? + pin + po)3” if F(n) = 
(n? + 1)3”. < 


Care must be taken when s = 1 when solving recurrence relations of the type covered by 
Theorem 6. In particular, to apply this theorem with F(n) = b,n, + b,n, ++: + bin + bo, 
the parameter s takes the value s = 1 (even though the term 1” does not explicitly appear). By 
the theorem, the form of the solution then depends on whether 1 is a root of the character- 
istic equation of the associated linear homogeneous recurrence relation. This is illustrated in 
Example 13, which shows how Theorem 6 can be used to find a formula for the sum of the first 
n positive integers. 


EXAMPLE 13 Leta, be the sum of the first n positive integers, so that 
n 
a= 5 k. 
k=1 
Note that a, satisfies the linear nonhomogeneous recurrence relation 
ay = An-1 +n. 
(To obtain a,,, the sum of the first n positive integers, from a,,_,, the sum of the firstn — 1 positive 
integers, we add n.) Note that the initial condition is a, = 1. 
The associated linear homogeneous recurrence relation for a,, is 
an = An-1 
The solutions of this homogeneous recurrence relation are given by a =¢(1) =c, 
where c is a constant. To find all solutions of a, = a,_, +n, we need find only a single par- 
ticular solution. By Theorem 6, because F(n) = n =n- (1)” and s = 1 is a root of degree one of 
the characteristic equation of the associated linear homogeneous recurrence relation, there is a 
particular solution of the form n(p,n + po) = pin? + pon. 
Inserting this into the recurrence relation gives pn? + Pon = p(n —- 1° + po(n—- 1) + 2. 
Simplifying, we see that n(2p, — 1) + (pọ — pı) = 0, which means that 2p, — 1 = 0 and pọ — 
Pı =9, so pọ = p; = 1/2. Hence, 
p 
if nin +1 
sn BEY 
n 2 2 2 
is a particular solution. Hence, all solutions of the original recurrence relation a, = a,_, +n 
are given by a, = a‘) +a” = c + n(n + 1)/2. Because a, = 1, we have 1 = a, =c+1-2/2= 
c+1,soc = Q. It follows that a, = n(n + 1)/2. (This is the same formula given in Table 2 in 
Section 2.4 and derived previously.) < 
Exercises 
1. Determine which of these are linear homogeneous recur- 2. Determine which of these are linear homogeneous recur- 
rence relations with constant coefficients. Also, find the rence relations with constant coefficients. Also, find the 
degree of those that are. degree of those that are. 
a) a, = 3a,_, +4a,_) +5a,_3 a) a, = 342 b) a, =3 
b) a, = 2na, + an2 c) a, = ap] + Ana S a, = ay d) a, = ap] + 24,3 
a,=a n 
d) n = An-] + 2 e) a, = a 4 + an-2 f) a, => a, + An-2 +nt 3 


f) ay = Ano 


g) a,=a,_, +n g) a, =4a,_)+5a,_4 + 9a,_7 


3. Solve these recurrence relations together with the initial 


conditions given. 
a) a, = 2a,_, forn > 1,a) =3 


b) a, =a,_, forn > 1, ay =2 

c) a, =5a,_; — 6a,_» for n > 2, ay = 1, a, =0 
d) a, = 4a,_,; — 4a,_, for n > 2, ay = 6, a, = 8 
e) a, = —4a,_, — 4a,_, for n > 2, aọ = 0, a, = 1 
f) a, =4a,_, forn > 2, a) =0,a, = 4 


g) a, =4,_,/4 forn > 2,4) =1, a, =0 

. Solve these recurrence relations together with the initial 
conditions given. 

a) a, =4d,_, + 6a,_, forn > 2, da) =3,a, =6 


b) a, =7a,_; — 10a,_, for n > 2, ap = 2, a, = 1 
c) da, = 6a,_, — 8a,_» for n > 2, ay = 4, a, = 10 
d) a, = 24,_) — 4,2 for n > 2, day = 4, a, = 1 

e) a, =4,_ forn > 2,dy =5,a, =—1 

f) a, = —6a,_, — 9d,_y for n > 2, dy = 3, a, = -3 


g) an2 = —4a,,, + 5a, for n > 0, ag = 2, a, = 8 

. How many different messages can be transmitted in n mi- 

croseconds using the two signals described in Exercise 19 

in Section 8.1? 

. How many different messages can be transmitted in n mi- 

croseconds using three different signals if one signal re- 

quires | microsecond for transmittal, the other two sig- 

nals require 2 microseconds each for transmittal, and a 

signal in a message is followed immediately by the next 

signal? 

. In how many ways can a 2 X n rectangular checkerboard 

be tiled using 1 x 2 and 2 x 2 pieces? 

. A model for the number of lobsters caught per year is 

based on the assumption that the number of lobsters 

caught in a year is the average of the number caught in 
the two previous years. 

a) Find a recurrence relation for {L,,}, where L, is the 
number of lobsters caught in year n, under the as- 
sumption for this model. 

b) Find L, if 100,000 lobsters were caught in year 1 and 
300,000 were caught in year 2. 

. A deposit of $100,000 is made to an investment fund at 

the beginning of a year. On the last day of each year two 

dividends are awarded. The first dividend is 20% of the 

amount in the account during that year. The second divi- 

dend is 45% of the amount in the account in the previous 

year. 

a) Find a recurrence relation for {P,,}, where P,, is the 
amount in the account at the end of n years if no 
money is ever withdrawn. 

b) How much is in the account after n years if no money 
has been withdrawn? 


* 10. Prove Theorem 2. 


11. The Lucas numbers satisfy the recurrence relation 


Links > 


Ln = Ly + Ly» 


and the initial conditions Lọ = 2 and L, = 1. 

a) Show that L, =f,_; +f,4, for n = 2, 3,..., where f, 
is the nth Fibonacci number. 

b) Find an explicit formula for the Lucas numbers. 


12. 


13. 


14. 


15. 


*16. 
17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


8.2 Solving Linear Recurrence Relations 551 


Find the solution to a, = 2a, +4,2- 2a,3 
for n = 3, 4, 5, ..., with ay = 3, a, = 6, and a, = 0. 


Find the solution to a, = 7a,_,+6a,_3; with dy = 9, 
a, = 10, and a, = 32. 

Find the solution to a, =5a,_, —4a,_4 with ap = 3, 
a; = 2, a = 6, and a; = 8. 

Find the solution to a, = 2a,_; +5a,_, — 6a,_3 with 
dy = 7, a, = —4, and a, = 8. 

Prove Theorem 3. 


Prove this identity relating the Fibonacci numbers and the 
binomial coefficients: 


foxy = Cn, 0) + Cn- 1,1) + + C(n— k, k), 


where n is a positive integer and k = |n/2]. [Hint: Let 
a, = C(n,0)+ Cin — 1, 1) + +++ Cn — k, k). Show that 
the sequence {a,,} satisfies the same recurrence relation 
and initial conditions satisfied by the sequence of Fi- 
bonacci numbers. | 


Solve the recurrence relation a, = 6a, _, — 12a,_.+ 
8a,_3 With ay = —5, a, = 4, and ay = 88. 

Solve the recurrence relation a, = —3a,_, — 3a,_, — 
d,-3 With dy = 5, a, = —9, and a, = 15. 

Find the general form of the solutions of the recurrence 


relation a, = 8a,_, — 16a,_4. 


What is the general form of the solutions of a linear ho- 
mogeneous recurrence relation if its characteristic equa- 
tion has roots 1, 1, 1, 1, —2, —2, —2, 3, 3, —4? 


What is the general form of the solutions of a linear ho- 
mogeneous recurrence relation if its characteristic equa- 
tion has the roots —1, —1, —1, 2, 2,5, 5, 7? 

Consider the nonhomogeneous linear recurrence relation 

a, = 3a,_1 +2”. 

a) Show that a, = —2”*! is a solution of this recurrence 
relation. 

b) Use Theorem 5 to find all solutions of this recurrence 
relation. 

c) Find the solution with ay = 1. 

Consider the nonhomogeneous linear recurrence relation 

a, = 2a,_, +2". 

a) Show that a, = n2” is a solution of this recurrence 
relation. 

b) Use Theorem 5 to find all solutions of this recurrence 
relation. 

c) Find the solution with ay = 2. 

a) Determine values of the constants A and B such that 
a, = An + B is a solution of recurrence relation a, = 
2a,_; tn+5. 

b) Use Theorem 5 to find all solutions of this recurrence 
relation. 


c) Find the solution of this recurrence relation with 
dy = 4. 
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26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


37. 


38. 


What is the general form of the particular solution 

guaranteed to exist by Theorem 6 of the linear non- 

homogeneous recurrence relation a„ = 6a,_, — 12a,_5 + 

8a,_3 + F(n) if 

a) F(n) = n°? 

c) F(n) = n2”? 

e) F(n) = n°2"? 

g) Fn) = 3? 

What is the general form of the particular solution guar- 

anteed to exist by Theorem 6 of the linear nonhomoge- 

neous recurrence relation a„ = 84,_ — 16a,_4 + F(n) if 

a) F(n) =n? b) F(n) = (—2)"? 

c) F(n) =n2"? d) F(n) = n°4"? 

e) F(n) = (n? — 2)(-2)"? f) F(n) = 42"? 

g) F(n) =2? 

a) Find all solutions 
a, = 2a,_; + 20°. 

b) Find the solution of the recurrence relation in part (a) 
with initial condition a, = 4. 

a) Find all solutions 
a, = 2a,_; + 3”. 

b) Find the solution of the recurrence relation in part (a) 
with initial condition a, = 5. 


b) F(n) = 2"? 
d) F(n) = (-2)"? 
£) F(n) = n3(-2)"? 


of the recurrence relation 


of the recurrence relation 


a) Find all solutions of the recurrence relation a, = 
—Sa,_; — 6a 2 + 42-4". 

b) Find the solution of this recurrence relation with a; = 
56 and a, = 278. 


Find all solutions of the recurrence relation a, = 
5a,_1 — 64,_3 + 2” + 3n. [Hint: Look for a particular so- 
lution of the form gn2" + p;n + p>, where q, p,, and py 
are constants. ] 

Find the solution of the recurrence relation a, = 
2a, t3: 2”. 

Find all solutions of the recurrence relation a, = 
4a,_, —4a,_. + (n+ 1)2". 

all solutions of the recurrence relation a, = 
— 16a,_5 + 12a,_3 +n4" with dy = -2, 
a, = 0, anda, = 5. 

Find the solution of the recurrence relation a, = 
4a,_; — 3d,_. + 2" +n +3 with ay = 1 and a, = 4. 


n 


Let a, be the sum of the first n perfect squares, that 
is, a, = yy Show that the sequence {a,} sat- 
isfies the linear nonhomogeneous recurrence relation 
a, = a„_ı +n’ and the initial condition a, = 1. Use The- 
orem 6 to determine a formula for a,, by solving this re- 
currence relation. 


Let a, be the sum of the first n triangular numbers, that 
is, a, = Dy _ ty, where t, = k(k + 1)/2. Show that {a,} 
satisfies the linear nonhomogeneous recurrence relation 
a, = a, +n(n+ 1)/2 and the initial condition a, = 1. 
Use Theorem 6 to determine a formula for a, by solving 


this recurrence relation. 

a) Find the characteristic roots of the linear ho- 
mogeneous recurrence relation a, = 2a,_; — 2d,_». 
[Note: These are complex numbers. ] 


* 39, 


* 40. 


* AL. 


42. 


43. 


* 44, 


45. 


46. 


b) Find the solution of the recurrence relation in part (a) 
with ay = Landa, =2. 

a) Find the characteristic roots of the linear homoge- 
neous recurrence relation a„ = a,,_4. Note: These in- 
clude complex numbers. ] 

b) Find the solution of the recurrence relation in part (a) 
with dy = 1, a, = 0, a, = —1, anda, = 1. 

Solve the simultaneous recurrence relations 

a, = 3a,_,; + 2b, 
b, = an-ı + 2Dy-1 


with dy = 1 and bọ = 2. 

a) Use the formula found in Example 4 for f,, the nth 
Fibonacci number, to show that f, is the integer 
closest to 


(4) 


b) Determine for which n f, is greater than 


(34) 


and for which n f, is less than 


(24) 


Show that if a, =4a,_,+4,_., a9 =s and a, =t, 
where s and f are constants, then a, = sf,,_, + tf, for all 
positive integers n. 


Express the solution of the linear nonhomogenous re- 
currence relation a, = d,_, + 4,2 + 1 for n > 2 where 
dy = 0 and a, = 1 in terms of the Fibonacci numbers. 
[Hint: Let b, = a, + 1 and apply Exercise 42 to the se- 
quence b,,.] 


(Linear algebra required) Let A, be the n Xn matrix 
with 2s on its main diagonal, 1s in all positions next to a 
diagonal element, and Os everywhere else. Find a recur- 
rence relation for d,, the determinant of A,,. Solve this 
recurrence relation to find a formula for d,,. 

Suppose that each pair of a genetically engineered species 

of rabbits left on an island produces two new pairs of rab- 

bits at the age of 1 month and six new pairs of rabbits at 
the age of 2 months and every month afterward. None of 
the rabbits ever die or leave the island. 

a) Find a recurrence relation for the number of pairs of 
rabbits on the island n months after one newborn pair 
is left on the island. 

b) By solving the recurrence relation in (a) determine 
the number of pairs of rabbits on the island n months 
after one pair is left on the island. 

Suppose that there are two goats on an island initially. 

The number of goats on the island doubles every year by 

natural reproduction, and some goats are either added or 

removed each year. 


47. 


a) Construct a recurrence relation for the number of 
goats on the island at the start of the nth year, assum- 
ing that during each year an extra 100 goats are put 
on the island. 

b) Solve the recurrence relation from part (a) to find the 


number of goats on the island at the start of the nth 


year. 
c) Construct a recurrence relation for the number of 


goats on the island at the start of the nth year, assum- 
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where b, = g(n + 1)Q(n + 1)a„, with 
O(n) = FASO) f(a = 1/88 = 8). 


b) Use part (a) to solve the original recurrence relation 
to obtain 


— C+ Yi QOD 
n= G+ DON + I) ` 


ing that n goats are removed during the nth year for *49. Use Exercise 48 to solve the recurrence relation 
each n > 3. (n+ la, = (n + 3)a,_, +n, forn > 1, with ay = 1. 
d) Solve the recurrence relation in part (c) for the num- 50. It can be shown that C,, the average number of com- 
n 


ber of goats on the island at the start of the nth year. 
A new employee at an exciting new software company 
starts with a salary of $50,000 and is promised that at the 
end of each year her salary will be double her salary of 
the previous year, with an extra increment of $10,000 for 


parisons made by the quick sort algorithm (described in 
preamble to Exercise 50 in Section 5.4), when sorting n 
elements in random order, satisfies the recurrence rela- 
tion 


each year she has been with the company. 
a) Construct a recurrence relation for her salary for her 


n=1 


C =n+1+2 5G 
nN =o 


nth year of employment. 


b) Solve this recurrence relation to find her salary for her 
nth year of employment. 


Some linear recurrence relations that do not have constant co- 
efficients can be systematically solved. This is the case for re- 
currence relations of the form f(m)a, = g(n)a,_, + h(n). Ex- 
ercises 48—50 illustrate this. 

* 48. a) Show that the recurrence relation 


f(a, = gína, F h(n), 53. 


for n > 1, and with ay = C, can be reduced to a recur- 
rence relation of the form 


b, = Dy + OA(njh(n), 


forn = 1, 2,..., with initial condition Cy = 0. 
a) Show that {C,,} also satisfies the recurrence relation 
nC, =(n+1)C,_; + 2nforn = 1,2,.... 
b) Use Exercise 48 to solve the recurrence relation in 
part (a) to find an explicit formula for C,,. 
** 51, Prove Theorem 4. 
** 52. Prove Theorem 6. 
Solve the recurrence relation T(n) = nT?(n/2) with ini- 
tial condition T(1) = 6 when n = 2* for some integer k. 
[Hint: Let n = 2* and then make the substitution a, = 
log T(2") to obtain a linear nonhomogeneous recurrence 
relation. | 


Divide-and-Conquer Algorithms and Recurrence Relations 


Links » 


“Divide et impera” 
(translation: “Divide 
and conquer”) —Julius 
Caesar 


8.3.1 Introduction 


Many recursive algorithms take a problem with a given input and divide it into one or more 
smaller problems. This reduction is successively applied until the solutions of the smaller prob- 
lems can be found quickly. For instance, we perform a binary search by reducing the search for 
an element in a list to the search for this element in a list half as long. We successively apply 
this reduction until one element is left. When we sort a list of integers using the merge sort, we 
split the list into two halves of equal size and sort each half separately. We then merge the two 
sorted halves. Another example of this type of recursive algorithm is a procedure for multiplying 
integers that reduces the problem of the multiplication of two integers to three multiplications 
of pairs of integers with half as many bits. This reduction is successively applied until integers 
with one bit are obtained. There procedures follow an important algorithmic paradigm known 
as divide-and-conquer, and are called divide-and-conquer algorithms, because they divide 
a problem into one or more instances of the same problem of smaller size and they conquer 
the problem by using the solutions of the smaller problems to find a solution of the original 
problem, perhaps with some additional work. 

In this section we will show how recurrence relations can be used to analyze the compu- 
tational complexity of divide-and-conquer algorithms. We will use these recurrence relations 
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EXAMPLE 1 


Extra » 
Examples 


EXAMPLE 2 


EXAMPLE 3 


to estimate the number of operations used by many different divide-and-conquer algorithms, 
including several that we introduce in this section. 


8.3.2 Divide-and-Conquer Recurrence Relations 


Suppose that a recursive algorithm divides a problem of size n into a subproblems, where each 
subproblem is of size n/b (for simplicity, assume that n is a multiple of b; in reality, the smaller 
problems are often of size equal to the nearest integers either less than or equal to, or greater 
than or equal to, n/b). Also, suppose that a total of g(n) extra operations are required in the 
conquer step of the algorithm to combine the solutions of the subproblems into a solution of 
the original problem. Then, if f(n) represents the number of operations required to solve the 
problem of size n, it follows that f satisfies the recurrence relation 


fn) = af(n/b) + gn). 


This is called a divide-and-conquer recurrence relation. 

We will first set up the divide-and-conquer recurrence relations that can be used to study 
the complexity of some important algorithms. Then we will show how to use these divide-and- 
conquer recurrence relations to estimate the complexity of these algorithms. 


Binary Search We introduced a binary search algorithm in Section 3.1. This binary search 
algorithm reduces the search for an element in a search sequence of size n to the binary search 
for this element in a search sequence of size n/2, when n is even. (Hence, the problem of size n 
has been reduced to one problem of size n/2.) Two comparisons are needed to implement this 
reduction (one to determine which half of the list to use and the other to determine whether any 
terms of the list remain). Hence, if f (n) is the number of comparisons required to search for an 
element in a search sequence of size n, then 


f(y) =f(a/2) +2 


when n is even. < 


Finding the Maximum and Minimum of a Sequence Consider the following algorithm for 
locating the maximum and minimum elements of a sequence a), z, ..., a,. If n = 1, then a, is 
the maximum and the minimum. If n > 1, split the sequence into two sequences, either where 
both have the same number of elements or where one of the sequences has one more element 
than the other. The problem is reduced to finding the maximum and minimum of each of the 
two smaller sequences. The solution to the original problem results from the comparison of the 
separate maxima and minima of the two smaller sequences to obtain the overall maximum and 
minimum. 

Let f(n) be the total number of comparisons needed to find the maximum and minimum 
elements of the sequence with n elements. We have shown that a problem of size n can be 
reduced into two problems of size n/2, when n is even, using two comparisons, one to compare 
the maxima of the two sequences and the other to compare the minima of the two sequences. 
This gives the recurrence relation 


fn) = 2f(n/2) +2 


when n is even. < 


Merge Sort The merge sort algorithm (introduced in Section 5.4) splits a list to be sorted 
with n items, where n is even, into two lists with n/2 elements each, and uses fewer than n 


EXAMPLE 4 


Links » 


EXAMPLE 5 


Links > 
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comparisons to merge the two sorted lists of n/2 items each into one sorted list. Consequently, 
the number of comparisons used by the merge sort to sort a list of n elements is less than M(n), 
where the function M(n) satisfies the divide-and-conquer recurrence relation 


M(n) = 2M(n/2) +n. < 


Fast Multiplication of Integers Surprisingly, there are more efficient algorithms than the 
conventional algorithm (described in Section 4.2) for multiplying integers. One of these algo- 
rithms, which uses a divide-and-conquer technique, will be described here. This fast multipli- 
cation algorithm proceeds by splitting each of two 2n-bit integers into two blocks, each with n 
bits. Then, the original multiplication is reduced from the multiplication of two 2n-bit integers 
to three multiplications of n-bit integers, plus shifts and additions. 

Suppose that a and b are integers with binary expansions of length 2n (add initial bits of 
zero in these expansions if necessary to make them the same length). Let 


A = (Ayn 1 4yn-2 *** 4140)2 and b = (ban-1b2n-2 *** Oy D0)2- 
Let 

a=2"A,+Apj, b=2"B, + Bo, 
where 


Ay = Gopi 7° an1 4n) Ag = Gy-1 7° 4140)» 


B; = (ban-1 an Dns Ondo» Bo = (bni _ bby). 


The algorithm for fast multiplication of integers is based on the fact that ab can be 
rewritten as 


ab = (27" + 2")A,B, + 2"(A, — Ap)(By — By) + (2 + DApBo- 


The important fact about this identity is that it shows that the multiplication of two 2n-bit in- 
tegers can be carried out using three multiplications of n-bit integers, together with additions, 
subtractions, and shifts. This shows that if f(n) is the total number of bit operations needed to 
multiply two n-bit integers, then 


f(2n) = 3f(n) + Cn. 


The reasoning behind this equation is as follows. The three multiplications of n-bit integers 
are carried out using 3f(n)-bit operations. Each of the additions, subtractions, and shifts uses a 
constant multiple of n-bit operations, and Cn represents the total number of bit operations used 
by these operations. < 


Fast Matrix Multiplication In Example 7 of Section 3.3 we showed that multiplying two n x 
n matrices using the definition of matrix multiplication required n? multiplications and n?(n — 
1) additions. Consequently, computing the product of two n x n matrices in this way requires 
O(n*) operations (multiplications and additions). Surprisingly, there are more efficient divide- 
and-conquer algorithms for multiplying two n Xn matrices. Such an algorithm, invented by 
Volker Strassen in 1969, reduces the multiplication of two n x n matrices, when n is even, to 
seven multiplications of two (n/2) x (n/2) matrices and 15 additions of (n/2) x (n/2) matrices. 
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(See [CoLeRiSt09] for the details of this algorithm.) Hence, if f(n) is the number of operations 
(multiplications and additions) used, it follows that 


f(n) = Tf(n/2) + 15n? /4 


when n is even. < 


As Examples 1-5 show, recurrence relations of the form f(n) = af(n/b) + g(n) arise in 
many different situations. It is possible to derive estimates of the size of functions that satisfy 
such recurrence relations. Suppose that f satisfies this recurrence relation whenever n is divisible 
by b. Let n = b*, where k is a positive integer. Then 


f(n) = af (n/b) + g(n) 
= af (n/b’) + ag(n/b) + g(n) 
= af (n/b>) + a’g(n/b’) + ag(n/b) + g(n) 


k-1 


= alf(n/b') + ¥' aig(n/b’). 


j=0 
Because n/b* = 1, it follows that 


k-1 


fn) = af) + X a'g(n/b’). 


j=0 


We can use this equation for f(n) to estimate the size of functions that satisfy divide-and-conquer 
relations. 


THEOREM 1 Letf be an increasing function that satisfies the recurrence relation 


f(n) = af(n/b) +c 


whenever n is divisible by b, where a > 1, b is an integer greater than 1, and c is a positive 
real number. Then 


— (OIH if a> 1, 
f(n) is ; 
O(logn) ifa=1. 


Furthermore, when n = b‘and a # 1, where kis a positive integer, 
fn) = Cnet + ©, 
where C, = f(1) +c/(a— 1) and C, = —c/ (a — 1). 


Proof: First let n = bt. From the expression for f(n) obtained in the discussion preceding the 
theorem, with g(n) = c, we have 


k-1 k-1 


fin) =a'f() + Y c= dfc Dia’. 


j=0 j=0 


EXAMPLE 6 
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Examples 


» 
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When a = 1 we have 
f(r) =f) + ck. 


Because n = b*, we have k = log, n. Hence, 


fm) =f) + clog,n. 


When n is not a power of b, we have bk < n < b‘*!, for a positive integer k. Because f is increas- 
ing, it follows that f(n) < f(W!) =f) + ck + 1) = (fd) +0) +ck < (fd) +0) + clog, n. 
Therefore, in both cases, f(n) is O(log n) when a = 1. 

Now suppose that a > 1. First assume that n = b*, where k is a positive integer. From the 
formula for the sum of terms of a geometric progression (Theorem 1 in Section 2.4), it follows 
that 


fn) = af) + ca — 1)/(a — 1) 
= a‘(f(1) + c/(a-1)] -—c/(@-1) 
= Cnt" + C, 
because a‘ = q!°8" = n!% 4 (see Exercise 4 in Appendix 2), where C, = f(1) + c/(a — 1) and 
C= =e/@= 1). 


Now suppose that n is not a power of b. Then bt < n < b*+!, where k is a nonnegative 
integer. Because f is increasing, 


f@) <f) = Ca" +, 
< (C,a)a!%" + C, 


= (C,a)n'% 4 +C, 


because k < logn < k+ 1. 
Hence, we have f(n) is O(m% 9). < 


Examples 6-9 illustrate how Theorem 1 is used. 


Let f(n) = 5f (n/2) + 3 and f(1) = 7. Find f(2"), where k is a positive integer. Also, estimate 
f(n) iff is an increasing function. 
Solution: From the proof of Theorem 1, with a = 5, b = 2, and c = 3, we see that if n = 2* then 
F) = dF) + e/@—-1)) + [-c/(@- 1)] 
= 5*(7 + (3/4)] - 3/4 
= 5*(31/4) — 3/4. 


Also, if f(n) is increasing, Theorem 1 shows that f(n) is O(n!°%*) = O(n!85). < 
We can use Theorem | to estimate the computational complexity of the binary search al- 


gorithm and the algorithm given in Example 2 for locating the minimum and maximum of a 
sequence. 
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EXAMPLE 7 


EXAMPLE 8 


THEOREM 2 


EXAMPLE 9 


EXAMPLE 10 


Give a big-O estimate for the number of comparisons used by a binary search. 


Solution: In Example 1 it was shown that f(n) = f(n/2) +2 when n is even, where f is the 
number of comparisons required to perform a binary search on a sequence of size n. Hence, 
from Theorem 1, it follows that f(n) is O(log n). < 


Give a big-O estimate for the number of comparisons used to locate the maximum and minimum 
elements in a sequence using the algorithm given in Example 2. 


Solution: In Example 2 we showed that f(n) = 2f(n/2) + 2, when n is even, where f is the 
number of comparisons needed by this algorithm. Hence, from Theorem 1, it follows that f(n) 
is O(n'’2?) = O(n). < 


We now state a more general, and more complicated, theorem, which has Theorem 1 as a 
special case. This theorem (or more powerful versions, including big-Theta estimates) is some- 
times known as the master theorem because it is useful in analyzing the complexity of many 
important divide-and-conquer algorithms. 


MASTER THEOREM Let f be an increasing function that satisfies the recurrence 
relation 


f(n) = af(n/b) + cn? 


whenever n = b¥, where k is a positive integer, a > 1, b is an integer greater than 1, and c 
and d are real numbers with c positive and d nonnegative. Then 


O(n?) je pace (oe 
f(n) is? O(n“ logn) if a = bf, 
O(n) ifa> bt. 


The proof of Theorem 2 is left for the reader as Exercises 29-33. 


Complexity of Merge Sort In Example 3 we explained that the number of comparisons used 
by the merge sort to sort a list of n elements is less than M(n), where M(n) = 2M(n/2) + n. By 
the master theorem (Theorem 2) we find that M(n) is O(n log n), which agrees with the estimate 
found in Section 5.4. < 


Give a big-O estimate for the number of bit operations needed to multiply two n-bit integers 
using the fast multiplication algorithm described in Example 4. 


Solution: Example 4 shows that f(n) = 3f(n/2) + Cn, when n is even, where f (n) is the number 
of bit operations required to multiply two n-bit integers using the fast multiplication algorithm. 
Hence, from the master theorem (Theorem 2), it follows that f(n) is O(n'°2). Note that log 
3 ~ 1.6. Because the conventional algorithm for multiplication uses O(n’) bit operations, the 
fast multiplication algorithm is a substantial improvement over the conventional algorithm in 


EXAMPLE 11 


EXAMPLE 12 


Links » 


It took researchers more 
than 10 years to find 

an algorithm with 

O(n log n) complexity 
that locates the closest 
pair of points among n 
points. 
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terms of time complexity for sufficiently large integers, including large integers that occur in 
practical applications. < 


Give a big-O estimate for the number of multiplications and additions required to multiply two 
n X n matrices using the matrix multiplication algorithm referred to in Example 5. 


Solution: Let f(n) denote the number of additions and multiplications used by the algorithm 
mentioned in Example 5 to multiply two n x n matrices. We have f(n) = 7f(n/2) + 15n? /4, 
when n is even. Hence, from the master theorem (Theorem 2), it follows that f(n) is O(n'°2’). 
Note that log 7 ~ 2.8. Because the conventional algorithm for multiplying two n x n matrices 
uses O(n?) additions and multiplications, it follows that for sufficiently large integers n, includ- 
ing those that occur in many practical applications, this algorithm is substantially more efficient 
in time complexity than the conventional algorithm. < 


THE CLOSEST-PAIR PROBLEM We conclude this section by introducing a divide-and- 
conquer algorithm from computational geometry, the part of discrete mathematics devoted to 
algorithms that solve geometric problems. 


The Closest-Pair Problem Consider the problem of determining the closest pair of points in 
a set of n points (x1, Y1) ..., Xw Yn) in the plane, where the distance between two points (x; y;) 


and (x, y;) is the usual Euclidean distance 4/ (x; — x +t- yj). This problem arises in many 


applications such as determining the closest pair of airplanes in the air space at a particular 
altitude being managed by an air traffic controller. How can this closest pair of points be found 
in an efficient way? 


Solution: To solve this problem we can first determine the distance between every pair of points 
and then find the smallest of these distances. However, this approach requires O(n?) compu- 
tations of distances and comparisons because there are C(n, 2) = n(n — 1)/2 pairs of points. 
Surprisingly, there is an elegant divide-and-conquer algorithm that can solve the closest-pair 
problem for n points using O(n logn) computations of distances and comparisons. The algo- 
rithm we describe here is due to Michael Samos (see [PrSa85]). 

For simplicity, we assume that n = 2*, where k is a positive integer. (We avoid some tech- 
nical considerations that are needed when n is not a power of 2.) When n = 2, we have only 
one pair of points; the distance between these two points is the minimum distance. At the start 
of the algorithm we use the merge sort twice, once to sort the points in order of increasing x 
coordinates, and once to sort the points in order of increasing y coordinates. Each of these sorts 
requires O(n log n) operations. We will use these sorted lists in each recursive step. 

The recursive part of the algorithm divides the problem into two subproblems, each involv- 
ing half as many points. Using the sorted list of the points by their x coordinates, we construct a 
vertical line 7 dividing the n points into two parts, a left part and a right part of equal size, each 
containing n/2 points, as shown in Figure 1. (If any points fall on the dividing line 7, we divide 
them among the two parts if necessary.) At subsequent steps of the recursion we need not sort 
on x coordinates again, because we can select the corresponding sorted subset of all the points. 
This selection is a task that can be done with O(n) comparisons. 

There are three possibilities concerning the positions of the closest points: (1) they are both 
in the left region L, (2) they are both in the right region R, or (3) one point is in the left region 
and the other is in the right region. Apply the algorithm recursively to compute d; and dp, 
where dz is the minimum distance between points in the left region and dp is the minimum 
distance between points in the right region. Let d = min(d,, dg). To successfully divide the 
problem of finding the closest two points in the original set into the two problems of finding the 
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In this illustration the problem of finding the 


d, closest pair in a set of 16 points is reduced to 
dp two problems of finding the closest pair in 


a set of eight points and the problem of 


closest determining whether there are points closer 
å pair than d = min(d;, dr) within the strip of 
width 2d centered at f. 
° 
L R 


FIGURE 1 The recursive step of the algorithm for solving the closest-pair problem. 


shortest distances between points in the two regions separately, we have to handle the conquer 
part of the algorithm, which requires that we consider the case where the closest points lie in 
different regions, that is, one point is in L and the other in R. Because there is a pair of points 
at distance d where both points lie in R or both points lie in L, for the closest points to lie in 
different regions requires that they must be a distance less than d apart. 

For a point in the left region and a point in the right region to lie at a distance less than d 
apart, these points must lie in the vertical strip of width 2d that has the line 7 as its center. (Oth- 
erwise, the distance between these points is greater than the difference in their x coordinates, 
which exceeds d.) To examine the points within this strip, we sort the points so that they are listed 
in order of increasing y coordinates, using the sorted list of the points by their y coordinates. 
At each recursive step, we form a subset of the points in the region sorted by their y coordi- 
nates from the already sorted set of all points sorted by their y coordinates, which can be done 
with O(n) comparisons. 

Beginning with a point in the strip with the smallest y coordinate, we successively examine 
each point in the strip, computing the distance between this point and all other points in the 
strip that have larger y coordinates that could lie at a distance less than d from this point. Note 
that to examine a point p, we need only consider the distances between p and points in the set 
that lie within the rectangle of height d and width 2d with p on its base and with vertical sides 
at distance d from 7. 

We can show that there are at most eight points from the set, including p, in or on this 2d x d 
rectangle. To see this, note that there can be at most one point in each of the eight d/2 x d/2 
squares shown in Figure 2. This follows because the farthest apart points can be on or within 


© one of these squares is the diagonal length d/ V2 (which can be found using the Pythagorean 
theorem), which is less than d, and each of these d/2 x d/2 squares lies entirely within the left 
region or the right region. This means that at this stage we need only compare at most seven 
distances, the distances between p and the seven or fewer other points in or on the rectangle, 
with d. 

Because the total number of points in the strip of width 2d does not exceed n (the total num- 
ber of points in the set), at most 7n distances need to be compared with d to find the minimum 
distance between points. That is, there are only 7n possible distances that could be less than d. 
Consequently, once the merge sort has been used to sort the pairs according to their x coordi- 
nates and according to their y coordinates, we find that the increasing function f(n) satisfying 
the recurrence relation 


fn) = 2f(n/2) + 7n, 
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d/2 d/2 d/2 d/2 
d/2 
At most eight points, including p, 
can lie in or on the 2d x d rectangle 
centered at € because at most one 
point can lie in or on each of the 
d/2 eight (d/2) x (d/2) squares. 
Ma 8 q 
° 
p 


FIGURE 2 Showing that there are at most seven other points to consider for each point 


in the strip. 


where f(2) = 1, exceeds the number of comparisons needed to solve the closest-pair problem 
for n points. By the master theorem (Theorem 2), it follows that f (n) is O(n log n). The two sorts 
of points by their x coordinates and by their y coordinates each can be done using O(n log n) 
comparisons, by using the merge sort, and the sorted subsets of these coordinates at each of the 
O(log n) steps of the algorithm can be done using O(n) comparisons each. Thus, we find that 


the closest-pair problem can be solved using O(n log n) comparisons. 


Exercises 


. How many comparisons are needed for a binary search 
in a set of 64 elements? 

. How many comparisons are needed to locate the max- 
imum and minimum elements in a sequence with 128 
elements using the algorithm in Example 2? 

. Multiply (1110), and (1010), using the fast multiplica- 
tion algorithm. 

. Express the fast multiplication algorithm in pseudocode. 
. Determine a value for the constant C in Example 4 and 
use it to estimate the number of bit operations needed to 
multiply two 64-bit integers using the fast multiplication 
algorithm. 

. How many operations are needed to multiply two 32 x 32 
matrices using the algorithm referred to in Example 5? 

. Suppose that f(n) = f(n/3) + 1 when nis a positive inte- 
ger divisible by 3, and f(1) = 1. Find 

a) f(3). b) f27). c) f(729). 

. Suppose that f(n) = 2f(n/2) + 3 when n is an even posi- 
tive integer, and f(1) = 5. Find 

a) (2). DFO AF6. D f(1024). 
. Suppose that f(n) = f(n/5) + 3n? when n is a positive in- 
teger divisible by 5, and f(1) = 4. Find 

a) f(5). b) f(125). c) f(3125). 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


< 


Find f(n) when n = 2%, where f satisfies the recurrence 
relation f(n) = f(n/2) + 1 with f(1) = 1. 


Give a big-O estimate for the function f in Exercise 10 if 
f is an increasing function. 


Find f(n) when n = 3‘, where f satisfies the recurrence 
relation f(n) = 2f(n/3) + 4 with f(1) = 1. 


Give a big-O estimate for the function f in Exercise 12 if 
f is an increasing function. 


Suppose that there are n = 2* teams in an elimination 
tournament, where there are n/2 games in the first round, 
with the n/2 = 2%! winners playing in the second round, 
and so on. Develop a recurrence relation for the number 
of rounds in the tournament. 


How many rounds are in the elimination tournament de- 
scribed in Exercise 14 when there are 32 teams? 


Solve the recurrence relation for the number of rounds in 
the tournament described in Exercise 14. 


Suppose that the votes of n people for different candi- 
dates (where there can be more than two candidates) for 
a particular office are the elements of a sequence. A per- 
son wins the election if this person receives a majority of 
the votes. 
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18. 


19. 


20. 


21. 


22. 


** 23, 


a) Devise a divide-and-conquer algorithm that deter- 
mines whether a candidate received a majority and, 
if so, determine who this candidate is. [Hint: Assume 
that n is even and split the sequence of votes into two 
sequences, each with n/2 elements. Note that a candi- 
date could not have received a majority of votes with- 
out receiving a majority of votes in at least one of the 
two halves.] 


b) Use the master theorem to give a big-O estimate for 
the number of comparisons needed by the algorithm 
you devised in part (a). 

Suppose that each person in a group of n people votes for 

exactly two people from a slate of candidates to fill two 

positions on a committee. The top two finishers both win 
positions as long as each receives more than n/2 votes. 

a) Devise a divide-and-conquer algorithm that deter- 
mines whether the two candidates who received the 
most votes each received at least n/2 votes and, if so, 
determine who these two candidates are. 


b) Use the master theorem to give a big-O estimate for 
the number of comparisons needed by the algorithm 
you devised in part (a). 

a) Set up a divide-and-conquer recurrence relation for 
the number of multiplications required to compute x”, 
where x is areal number and n is a positive integer, using 
the recursive algorithm from Exercise 26 in Section 5.4. 


b) Use the recurrence relation you found in part (a) to 
construct a big-O estimate for the number of multi- 
plications used to compute x” using the recursive al- 
gorithm. 


a) Set up a divide-and-conquer recurrence relation for 
the number of modular multiplications required to 
compute a” mod m, where a, m, and n are pos- 
itive integers, using the recursive algorithms from 
Example 4 in Section 5.4. 


b) Use the recurrence relation you found in part (a) to 
construct a big-O estimate for the number of modular 
multiplications used to compute a” mod m using the 
recursive algorithm. 


Suppose that the function f satisfies the recurrence re- 

lation f(n) = 2f (yn) + 1 whenever n is a perfect square 

greater than 1 and f (2) = 1. 

a) Find f(16). 

b) Give a big-O estimate for f(n). [Hint: Make the sub- 
stitution m = log n.] 

Suppose that the function f satisfies the recurrence rela- 

tion f(n) = 2f( yn) + log n whenever n is a perfect square 

greater than 1 and f (2) = 1. 

a) Find f(16). 

b) Find a big-O estimate for f(n). [Hint: Make the sub- 
stitution m = log n.] 

This exercise deals with the problem of finding the 

largest sum of consecutive terms of a sequence of n real 


24. 


25. 


*26. 


27. 


*28. 


numbers. When all terms are positive, the sum of all 
terms provides the answer, but the situation is more com- 
plicated when some terms are negative. For example, 
the maximum sum of consecutive terms of the sequence 
—2, 3, —1, 6, —7, 4 is 3 + (—1) + 6 = 8. (This exercise is 
based on [Be86].) Recall that in Exercise 56 in Section 
8.1 we developed a dynamic programming algorithm for 
solving this problem. Here, we first look at the brute-force 
algorithm for solving this problem; then we develop a 
divide-and-conquer algorithm for solving it. 


a) Use pseudocode to describe an algorithm that solves 
this problem by finding the sums of consecutive terms 
starting with the first term, the sums of consecutive 
terms starting with the second term, and so on, keep- 
ing track of the maximum sum found so far as the 
algorithm proceeds. 

b) Determine the computational complexity of the algo- 
rithm in part (a) in terms of the number of sums com- 
puted and the number of comparisons made. 

c) Devise a divide-and-conquer algorithm to solve this 
problem. [Hint: Assume that there are an even num- 
ber of terms in the sequence and split the sequence 
into two halves. Explain how to handle the case when 
the maximum sum of consecutive terms includes 
terms in both halves. ] 

d) Use the algorithm from part (c) to find the maximum 
sum of consecutive terms of each of the sequences: 
—2, 4,-1, 3, 5,—6, 1, 2;4, 1,—3, 7,-1,-5, 3, —2; and 

1, 6, 3, —4, —5, 8, —1, 7. 

e) Find a recurrence relation for the number of sums 
and comparisons used by the divide-and-conquer al- 
gorithm from part (c). 

f) Use the master theorem to estimate the computational 
complexity of the divide-and-conquer algorithm. 
How does it compare in terms of computational com- 
plexity with the algorithm from part (a)? 


Apply the algorithm described in Example 12 for finding 
the closest pair of points, using the Euclidean distance be- 
tween points, to find the closest pair of the points (1, 3), 
(1, 7), (2, 4), (2, 9), (3, 1), (3, 5), (4, 3), and (4, 7). 

Apply the algorithm described in Example 12 for finding 
the closest pair of points, using the Euclidean distance be- 
tween points, to find the closest pair of the points (1, 2), 
(1, 6), (2,4), (2,8), 3, 1), (3,6), (3, 10), (4, 3), (5, 1), 
(5, 5), (5, 9), (6, 7), (7, 1), (7, 4), (7, 9), and (8, 6). 

Use pseudocode to describe the recursive algorithm for 
solving the closest-pair problem as described in Exam- 
ple 12. 

Construct a variation of the algorithm described in Ex- 
ample 12 along with justifications of the steps used by 
the algorithm to find the smallest distance between two 
points if the distance between two points is defined to be 
d((x;, y;), (x; y;)) = max(|x; =x;l, ly; = y;l). 

Suppose someone picks a number x from a set of n 
numbers. A second person tries to guess the number 
by successively selecting subsets of the n numbers and 
asking the first person whether x is in each set. The 
first person answers either “yes” or “no.” When the first 
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person answers each query truthfully, we can find x 
using log n queries by successively splitting the sets 
used in each query in half. Ulam’s problem, proposed by 
Stanislaw Ulam in 1976, asks for the number of queries 
required to find x, supposing that the first person is al- 
lowed to lie exactly once. 


a) Show that by asking each question twice, given a 
number x and a set with n elements, and asking one 
more question when we find the lie, Ulam’s problem 
can be solved using 2logn + 1 queries. 

b) Show that by dividing the initial set of n elements 
into four parts, each with n/4 elements, 1/4 of the ele- 
ments can be eliminated using two queries. [Hint: Use 
two queries, where each of the queries asks whether 
the element is in the union of two of the subsets with 
n/4 elements and where one of the subsets of n/4 el- 
ements is used in both queries. ] 

c) Show from part (b) that if f(n) equals the number 
of queries used to solve Ulam’s problem using the 
method from part (b) and n is divisible by 4, then 
fm) =fGBn/4) + 2. 

d) Solve the recurrence relation in part (c) for f(n). 

e) Is the naive way to solve Ulam’s problem by ask- 
ing each question twice or the divide-and-conquer 
method based on part (b) more efficient? The most 
efficient way to solve Ulam’s problem has been 
determined by A. Pelc [Pe87]. 
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In Exercises 29-33, assume that f is an increasing func- 
tion satisfying the recurrence relation f(n) = af(n/b) + cnf, 
where a > 1, b is an integer greater than 1, and c and d 


are 


positive real numbers. These exercises supply a proof of 


Theorem 2. 


* 29. 


30. 


*31. 


32. 


33. 


34. 


35. 


36. 


37. 


Show that if a = b? and n is a power of b, then f(n) = 
fn? + cn“ log, n. 


Use Exercise 29 to show that if a = bf, then f(n) is 
O(n“ logn). 

Show that if a # b? and n is a power of b, then f(n) = 
Cyn? + Cyn'®»*, where C, = bfc/(b! —a) and C, = 
fC) + b¢c/(a — b’). 


Use Exercise 31 to show that if a < bf, then f(n) is O(n”). 


Use Exercise 31 to show that if a> bf, then f(n) is 
O(n'°8»4), 

Find f(n) when n = 4“, where f satisfies the recurrence 
relation f(n) = 5f(n/4) + 6n, with f(1) = 1. 

Give a big-O estimate for the function f in Exercise 34 if 
f is an increasing function. 

Find f(n) when n = 2‘, where f satisfies the recurrence 
relation f(n) = 8f(n/2) + n? with f(1) = 1. 


Give a big-O estimate for the function f in Exercise 36 if 
f is an increasing function. 


8.4.1 Introduction 


Links » 


Generating functions are used to represent sequences efficiently by coding the terms of a se- 
quence as coefficients of powers of a variable x in a formal power series. Generating functions 


can be used to solve many types of counting problems, such as the number of ways to select 
or distribute objects of different kinds, subject to a variety of constraints, and the number of 
ways to make change for a dollar using coins of different denominations. Generating functions 
can be used to solve recurrence relations by translating a recurrence relation for the terms of 
a sequence into an equation involving a generating function. This equation can then be solved 
to find a closed form for the generating function. From this closed form, the coefficients of the 
power series for the generating function can be found, solving the original recurrence relation. 
Generating functions can also be used to prove combinatorial identities by taking advantage of 
relatively simple relationships between functions that can be translated into identities involving 
the terms of sequences. Generating functions are a helpful tool for studying many properties of 
sequences besides those described in this section, such as their use for establishing asymptotic 


formulae for the terms of a sequence. 


We begin with the definition of the generating function for a sequence. 


Definition 1 


The generating function for the sequence do, a, ..., Ap ... Of real numbers is the infinite 
series 
ce 
G(x) = dg tayx+--+ Gx ap ae = ` a 


k=0 
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EXAMPLE 1 


Extra » 
Examples 


EXAMPLE 2 


EXAMPLE 3 


Remark: The generating function for {a,} given in Definition 1 is sometimes called the ordi- 
nary generating function of {a,} to distinguish it from other types of generating functions for 
this sequence. 


The generating functions for the sequences {a,} with a,=3,aq,=k+1, and a, = 2" 


are DP 3 Dy ok + Lx", and X? o 2%, respectively. < 


We can define generating functions for finite sequences of real numbers by extending a 
finite sequence ap, a), ..., a, into an infinite sequence by setting a,,, = 0, a,,. = 0, and so on. 
The generating function G(x) of this infinite sequence {a,,} is a polynomial of degree n because 
no terms of the form ax! with j > n occur, that is, 


G(x) = dy tax + + + a,x”. 


What is the generating function for the sequence 1, 1, 1, 1, 1, 1? 
Solution: The generating function of 1, 1, 1, 1, 1, 1 is 
ltextr trex tr. 
By Theorem 1 of Section 2.4 we have 
Qf- 1)/œ&-1)=1+x+ 40 2° 42° 
when x#1. Consequently, G(x) = (x® — 1) /(x-1) is the generating function of the 


sequence 1, 1, 1, 1, 1, 1. [Because the powers of x are only place holders for the terms of the 
sequence in a generating function, we do not need to worry that G(1) is undefined. ] < 


Let m be a positive integer. Let a, = C(m, k), for k = 0, 1, 2, ..., m. What is the generating func- 
tion for the sequence dp, ay, ..., Am? 


Solution: The generating function for this sequence is 
G(x) = C(m, 0) + C(m, Ix + C(m, 2)x* + ++ + C(m, m)x". 


The binomial theorem shows that G(x) = (1 + x)”. < 


8.4.2 Useful Facts About Power Series 


When generating functions are used to solve counting problems, they are usually considered to 
be formal power series. As such, they are treated as algebraic objects; questions about their 
convergence are ignored. However, when formal power series are convergent, valid operations 
carry over to their use as formal power series. We will take advantage of the power series of 
particular functions around x = 0. These power series are unique and have a positive radius of 
convergence. Readers familiar with calculus can consult textbooks on this subject for details 
about power series, including the convergence of the series we use here. 


EXAMPLE 4 


EXAMPLE 5 


THEOREM 1 


EXAMPLE 6 
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We will now state some widely important facts about infinite series used when working 
with generating functions. These facts can be found in calculus texts. 


The function f(x) = 1/(1 — x) is the generating function of the sequence 1, 1, 1, 1, ..., because 
1d-x=1l+xtx+-- 
for |x| < 1. < 
The function f(x) = 1/(1 — ax) is the generating function of the sequence 1, a, a, a>, ..., be- 
cause 
1/(1 -ax)=1 ax ax” po 
when |ax| < 1, or equivalently, for |x| < 1/|a| fora # 0. < 


We also will need some results on how to add and how to multiply two generating functions. 
Proofs of these results can be found in calculus texts. 


Wet Gy— ae and eG) = ae) When 


Cc S k 
fQ+8@= Lath and fow F, (x ah) i 


k=0 k=0 


Remark: Theorem 1 is valid only for power series that converge in an interval, as all series 
considered in this section do. However, the theory of generating functions is not limited to such 
series. In the case of series that do not converge, the statements in Theorem 1 can be taken as 
definitions of addition and multiplication of generating functions. 


We will illustrate how Theorem 1 can be used with Example 6. 


Let f(x) = 1/(1 — x)’. Use Example 4 to find the coefficients ag, a), a, ... in the expansion 


FO = Ero ak. 
Solution: From Example 4 we see that 
1/d—-x)=1l4+xtxr tH. 
Hence, from Theorem 1, we have 
œ k œ 
/a-x+ => (x i) = Pkt pat. 


k=0 \j=0 k=0 
< 


Remark: This result also can be derived from Example 4 by differentiation. Taking derivatives is 
a useful technique for producing new identities from existing identities for generating functions. 
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To use generating functions to solve many important counting problems, we will need to 
apply the binomial theorem for exponents that are not positive integers. Before we state an 
extended version of the binomial theorem, we need to define extended binomial coefficients. 


Definition 2 Let u be a real number and k a nonnegative integer. Then the extended binomial coefficient 
(x) is defined by 


yN Saw Delo A e O 
J AN Ti ke = (0), 


EXAMPLE 7 Find the values of the extended binomial coefficients (7) and eer 


Solution: Taking u = —2 and k = 3 in Definition 2 gives us 


(2) _ 2-34) L a 


3 3! 
Similarly, taking u = 1/2 and k = 3 gives us 


ee _ 0/D0/2-)40/2-2 
3j) 3! 
= (1/2)(-1/2)(-3/2)/6 
= 1/16. 


Example 8 provides a useful formula for extended binomial coefficients when the top pa- 
rameter is a negative integer. It will be useful in our subsequent discussions. 


EXAMPLE 8 When the top parameter is a negative integer, the extended binomial coefficient can be expressed 
in terms of an ordinary binomial coefficient. To see that this is the case, note that 


by definition of extended binomial coefficient 


=—n\ _ (=n)\-n=1)«(—n=—rt 1) 
rj} r! 


(-1yn(n+ 1) (n+r- 1) : : 
= factoring out —1 from each term in the numerator 


r! 
(-1’'(+r—-Int+r—2)--n i ee 
a a by the commutative law for multiplication 
r! 
iG oe 
ae multiplying both the numerator and denominator 
rin- 1)! by (n— 1)! 
-1 
=(-1)’ « g ) by the definition of binomial coefficients 
r 
=(-1)C(n+r-1,r) using alternative notation for binomial 


coefficients. 
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We now state the extended binomial theorem. 


THEOREM2 THE EXTENDED BINOMIAL THEOREM Let x be areal number with |x| < 1 and let 
u be areal number. Then 


roy, (i) xÉ. 


k=0 


Theorem 2 can be proved using the theory of Maclaurin series. We leave its proof to the reader 
with a familiarity with this part of calculus. 


Remark: When u is a positive integer, the extended binomial theorem reduces to the binomial 
theorem presented in Section 6.4, because in that case (") =Oifk>u. 


Example 9 illustrates the use of Theorem 2 when the exponent is a negative integer. 


EXAMPLE 9 Find the generating functions for (1 +x)” and (1 — x)~”, where n is a positive integer, using 
the extended binomial theorem. 


Solution: By the extended binomial theorem, it follows that 


dt+ay"=) a xÉ. 


k=0 
Using Example 8, which provides a simple formula for ( 7) , we obtain 


o0 


+x)" = VEY Cin +k—-1, kx. 
k=0 


Replacing x by —x, we find that 


d—-x"= X C(n +k- 1, kx. 
k=0 < 


Table 1 presents a useful summary of some generating functions that arise frequently. 


Remark: Note that the second and third formulae in this table can be deduced from the first 
formula by substituting ax and x" for x, respectively. Similarly, the sixth and seventh formulae 
can be deduced from the fifth formula using the same substitutions. The tenth and eleventh can 
be deduced from the ninth formula by substituting —x and ax for x, respectively. Also, some 
of the formulae in this table can be derived from other formulae using methods from calculus 
(such as differentiation and integration). Students are encouraged to know the core formulae in 
this table (that is, formulae from which the others can be derived, perhaps the first, fourth, fifth, 
eighth, ninth, twelfth, and thirteenth formulae) and understand how to derive the other formulae 
from these core formulae. 
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TABLE 1 Useful Generating Functions. 


+x)" = £ C(n, k)x* 
k=0 


= 1 + C(n, 1)x + C(n, DZ ++ + x" 
(1 +ax)" = > C(n, kyakx* 


k=0 
= 1 + C(n, lax + C(n, 2)a2x? + + + .a"x" 


(1l+.x’)" = > C(n, k)x* C(n, k/r) if r | k; 0 otherwise 
k=0 


= 1 + C(n, 1)x" + C(n, 2)x” + +x" 


= Vit alte dete’ 1 if k < n; 0 otherwise 
k=0 


1 if r | k; O otherwise 


Cnt+k-1k=Cn+k—-1n-1) 
(-1Ik Cn +k-1,k) =(-D'Ct+k-1n-1) 


C(n+k-1, bat = Cn+k-1,n- 1)a 


2 x3 
tatata 
po — 1)! 
Ind +x) = > ( 2 x = | CD k 
k=1 


Note: The series for the last two generating functions can be found in most calculus books when power series are discussed. 


EXAMPLE 10 


Extra » 
Examples 


EXAMPLE 11 
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8.4.3 Counting Problems and Generating Functions 


Generating functions can be used to solve a wide variety of counting problems. In particular, 
they can be used to count the number of combinations of various types. In Chapter 6 we de- 
veloped techniques to count the r-combinations from a set with n elements when repetition 
is allowed and additional constraints may exist. Such problems are equivalent to counting the 
solutions to equations of the form 


ej tents +e, =C, 


where C is a constant and each e; is a nonnegative integer that may be subject to a specified 
constraint. Generating functions can also be used to solve counting problems of this type, as 
Examples 10-12 show. 


Find the number of solutions of 
ei +e, +e; = 17, 
where e,, e), and e} are nonnegative integers with 2 < e; < 5,3 < e, < 6, and 4 < e} <7. 


Solution: The number of solutions with the indicated constraints is the coefficient of x!” in the 
expansion of 


EHEHEHE tx 40 +) +0 42° 42’). 


This follows because we obtain a term equal to x!” in the product by picking a term in 
the first sum x“!, a term in the second sum x, and a term in the third sum x^, where the 
exponents e}, e3, and e, satisfy the equation e; + e, + e3 = 17 and the given constraints. 

It is not hard to see that the coefficient of x!” in this product is 3. Hence, there are 
three solutions. (Note that the calculating of this coefficient involves about as much work 
as enumerating all the solutions of the equation with the given constraints. However, the 
method that this illustrates often can be used to solve wide classes of counting problems with 
special formulae, as we will see. Furthermore, a computer algebra system can be used to 
do such computations.) < 


In how many different ways can eight identical cookies be distributed among three distinct 
children if each child receives at least two cookies and no more than four cookies? 


Solution: Because each child receives at least two but no more than four cookies, for each child 
there is a factor equal to 


0? +23 ++À) 


in the generating function for the sequence {c,,}, where c, is the number of ways to distribute n 
cookies. Because there are three children, this generating function is 


+E HÅP. 


We need the coefficient of x* in this product. The reason is that the xë terms in the expansion 
correspond to the ways that three terms can be selected, with one from each factor, that have 
exponents adding up to 8. Furthermore, the exponents of the term from the first, second, and 
third factors are the numbers of cookies the first, second, and third children receive, respectively. 
Computation shows that this coefficient equals 6. Hence, there are six ways to distribute the 
cookies so that each child receives at least two, but no more than four, cookies. <q 
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EXAMPLE 12 


EXAMPLE 13 


Use generating functions to determine the number of ways to insert tokens worth $1, $2, 
and $5 into a vending machine to pay for an item that costs r dollars in both the cases when 
the order in which the tokens are inserted does not matter and when the order does matter. (For 
example, there are two ways to pay for an item that costs $3 when the order in which the tokens 
are inserted does not matter: inserting three $1 tokens or one $1 token and a $2 token. When 
the order matters, there are three ways: inserting three $1 tokens, inserting a $1 token and then 
a $2 token, or inserting a $2 token and then a $1 token.) 


Solution: Consider the case when the order in which the tokens are inserted does not matter. 
Here, all we care about is the number of each token used to produce a total of r dollars. Because 
we can use any number of $1 tokens, any number of $2 tokens, and any number of $5 tokens, 
the answer is the coefficient of x” in the generating function 


txt te te dt? txt tot $0 +22 $c 4 o). 


(The first factor in this product represents the $1 tokens used, the second the $2 tokens used, 
and the third the $5 tokens used.) For example, the number of ways to pay for an item costing $7 
using $1, $2, and $5 tokens is given by the coefficient of x’ in this expansion, which equals 6. 

When the order in which the tokens are inserted matters, the number of ways to insert 
exactly n tokens to produce a total of r dollars is the coefficient of x” in 


(x + xX + x)", 


because each of the r tokens may be a $1 token, a $2 token, or a $5 token. Because any number 
of tokens may be inserted, the number of ways to produce r dollars using $1, $2, or $5 tokens, 
when the order in which the tokens are inserted matters, is the coefficient of x” in 


1 
1 — (x +x? +) 
— 
1—x—x2 — x5’ 


Lt tx? HDAS $e 


where we have added the number of ways to insert 0 tokens, 1 token, 2 tokens, 3 tokens, 
and so on, and where we have used the identity 1/(1 — x) = 1 +x +x? +++ with x replaced 
with x + x? + x°. For example, the number of ways to pay for an item costing $7 using $1, $2, 
and $5 tokens, when the order in which the tokens are used matters, is the coefficient of x’ in this 
expansion, which equals 26. [Hint: To see that this coefficient equals 26 requires the addition of 
the coefficients of x’ in the expansions (x + x? +x} for 2 < k < 7. This can be done by hand 
with considerable computation, or a computer algebra system can be used.] < 


Example 13 shows the versatility of generating functions when used to solve problems with 
differing assumptions. 


Use generating functions to find the number of k-combinations of a set with n elements. Assume 
that the binomial theorem has already been established. 


Solution: Each of the n elements in the set contributes the term (1 + x) to the generating function 


x)= _, a,x". Here f(x) is the generating function for {a,}, where a, represents the number 
) r04. H ) is the g ting function for {a,}, where a, represents th b 
of k-combinations of a set with n elements. Hence, 


f= a+". 


EXAMPLE 14 
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But by the binomial theorem, we have 


fx) = 5 (7) Pa 


k=0 


where 


n\ _ n! 
k) k\n—k! 


Hence, C(n, k), the number of k-combinations of a set with n elements, is 


n! 


n-A < 


Remark: We proved the binomial theorem in Section 6.4 using the formula for the number of 
r-combinations of a set with n elements. This example shows that the binomial theorem, which 
can be proved by mathematical induction, can be used to derive the formula for the number of 
r-combinations of a set with n elements. 


Use generating functions to find the number of r-combinations from a set with n elements when 
repetition of elements is allowed. 


Solution: Let G(x) be the generating function for the sequence {a,}, where a, equals the number 


of r-combinations of a set with n elements with repetitions allowed. That is, G(x) = YY, a,x". 


Because we can select any number of a particular member of the set with n elements when we 
form an r-combination with repetition allowed, each of the n elements contributes (1 + x +x? + 
xX? + +++) to a product expansion for G(x). Each element contributes this factor because it may 
be selected zero times, one time, two times, three times, and so on, when an r-combination is 
formed (with a total of r elements selected). Because there are n elements in the set and each 
contributes this same factor to G(x), we have 


Gx) = (14+ x42? +)". 
As long as |x| < 1, we have 1 +x +3? +++» = 1/(1 — x), so 
Gœ) = 1/(1 -x = (1 -x)™. 
Applying the extended binomial theorem (Theorem 2), it follows that 


a-9”=0 +" F (7) (=x. 


r=0 


The number of r-combinations of a set with n elements with repetitions allowed, when r is a 
positive integer, is the coefficient a, of x” in this sum. Consequently, using Example 8 we find 
that a, equals 


(;") (C17 = (-1)"C+r—-1,7n)-(-1" 


=C(n+r-1,r). < 


Note that the result in Example 14 is the same result we stated as Theorem 2 in Section 6.5. 
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EXAMPLE 15 Use generating functions to find the number of ways to select r objects of n different kinds if 
we must select at least one object of each kind. 


Solution: Because we need to select at least one object of each kind, each of the n kinds of objects 
contributes the factor (x +x? + x° + ---) to the generating function G(x) for the sequence {a,}, 
where a, is the number of ways to select r objects of n different kinds if we need at least one 
object of each kind. Hence, 


G(x) =F 404 He(Lt xt xe te = xA — x)". 
Using the extended binomial theorem and Example 8, we have 


G(x) = x"/(1 — x)" 


= x"-(1-x)™" 
n = =n r 
=y >( : Jew 


=x" VEY Cn $7 = 1,71 
r=0 


= > Cin+r-—1,r)x"t" 
r=0 


=) C¢-1,t-n)x 


t=n 


= y C(r—-1,r—n)x’. 


r=n 


We have shifted the summation in the next-to-last equality by setting t = n + r so that t =n 
when r = Qand n +r — 1 = t — 1, and then we replaced t by r as the index of summation in the 
last equality to return to our original notation. Hence, there are C(r — 1, r — n) ways to select r 
objects of n different kinds if we must select at least one object of each kind. 


8.4.4 Using Generating Functions to Solve 
Recurrence Relations 


We can find the solution to a recurrence relation and its initial conditions by finding an explicit 
formula for the associated generating function. This is illustrated in Examples 16 and 17. 


EXAMPLE 16 Solve the recurrence relation a, = 3a,_, for k = 1, 2,3, ... and initial condition ay = 2. 


Extra 
Examples » Solution: Let G(x) be the generating function for the sequence {a,}, that is, G(x) = bie a,x*. 
First note that 


co co 


xG(x) = > an = 5 NEn a 


k=0 k=1 


8.4 Generating Functions 


Using the recurrence relation, we see that 


co 


G(x) — 3xG(x) = > a,x* —3 > dyi" 


k=0 k=1 
oO 
k 
=a + > (a — 3a; )x 


k=1 


because dy = 2 and a, = 3a,_,. Thus, 


G(x) — 3xG(x) = (1 — 3x)G(x) = 2. 


573 


Solving for G(x) shows that G(x) = 2/(1 — 3x). Using the identity 1/(1 — ax) = Èg o aka, 


from Table 1, we have 


G(x) =2 D3 k> Deere, 
k=0 k=0 


Consequently, a, = 2 - 3%. 


EXAMPLE 17 Suppose that a valid codeword is an n-digit number in decimal notation containing an even 
number of Os. Let a, denote the number of valid codewords of length n. In Example 4 of Section 


8.1 we showed that the sequence {a, } satisfies the recurrence relation 
a, = 8a,_, + 10""" 


and the initial condition a, = 9. Use generating functions to find an explicit formula for a,,. 


Solution: To make our work with generating functions simpler, we extend this sequence 


by setting aọ = 1; when we assign this value to ad) and use the recurrence relation, 


we 


have a, = 8a) + 10° = 8 + 1 = 9, which is consistent with our original initial condition. (It also 


makes sense because there is one code word of length 0—the empty string.) 
We multiply both sides of the recurrence relation by x” to obtain 


a,x” = 8a,_,x" + 1077! x”. 


Let G(x) = Èo a,x" be the generating function of the sequence ap, a), az, .... We sum both 


sides of the last equation starting with n = 1, to find that 


foe) 


G@)-1= J ap" = )°(8a,_ x" + 10""!x") 


n=1 n=1 


œ œ 
=8 > dy—1X" + 5 ee 
n=l n=l 


œ foe] 
= 8x > ax! +x by Cy 
n=l n=1 
foe} foe} 
= 8x 5 a,X" +x 5 10”x” 
n=0 n=0 


= 8xG(x) + x/(1 — 10x), 
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EXAMPLE 18 


where we have used Example 5 to evaluate the second summation. Therefore, we have 
G(x) — 1 = 8xG(x) + x/(1 — 10x). 
Solving for G(x) shows that 


1 — 9x 
(1 — 8x)(1 — 10x)’ 


G(x) = 


Expanding the right-hand side of this equation into partial fractions (as is done in the integration 
of rational functions studied in calculus) gives 


G0) = 5 (Te + Ta): 


Using Example 5 twice (once with a = 8 and once with a = 10) gives 
1 oO o0 
G =- 8” n + 10” n 


=> TG +10")x", 


n=0 


Consequently, we have shown that 


1 
= =(8" + 10"). 
ee ) 


8.4.5 Proving Identities via Generating Functions 


In Chapter 6 we saw how combinatorial identities could be established using combinatorial 
proofs. Here we will show that such identities, as well as identities for extended binomial coef- 
ficients, can be proved using generating functions. Sometimes the generating function approach 
is simpler than other approaches, especially when it is simpler to work with the closed form 
of a generating function than with the terms of the sequence themselves. We illustrate how 
generating functions can be used to prove identities with Example 18. 


Use generating functions to show that 


> C(n, K = Cn, n) 
k=0 


whenever n is a positive integer. 


Solution: First note that by the binomial theorem C(2n, n) is the coefficient of x” in (1 + x”. 
However, we also have 


(1 +x)" =[(1 +x)"P 
= [C(n, 0) + C(n, 1)x + C(n, 2)? + + C(n, mx”. 


The coefficient of x” in this expression is 
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C(n, 0)C(n, n) + C(n, 1)C(n, n — 1) + C(n, 2)C(n, n — 2) + +++ + C(n, n)C(n, 0). 


This equals a C(n, k}, because C(n,n—k) = C(n, k). Because both C(2n,n) and 
pan C(n, k} represent the coefficient of x” in (1 + x)”, they must be equal. < 


Exercises 44 and 45 ask that Pascal’s identity and Vandermonde’s identity be proved using 


generating functions. 


Exercises 


1. Find the generating function for the finite sequence 2, 2, 
Ds D5 did 

2. Find the generating function for the finite sequence 1, 4, 
16, 64, 256. 


In Exercises 3-8, by a closed form we mean an algebraic ex- 
pression not involving a summation over a range of values or 
the use of ellipses. 


3. Find a closed form for the generating function for each of 
these sequences. (For each sequence, use the most obvi- 
ous choice of a sequence that follows the pattern of the 
initial terms listed.) 


a) 0,2, 2, 2, 2, 2, 2, 0, 0, 0, 0, 0, ... 
b) 0,0,0,1, 1, 1,1, 1,1,... 

c) 0, 1,0, 0, 1, 0,0, 1,0, 0, 1, ... 
d) 2,4, 8, 16, 32, 64, 128, 256, ... 


7 7 7 7 
o (2).(?).(Qon(7).0.00.00.. 


f) 2,-2,2, —2, 2, —2, 2, —2, ... 
g) 1,1,0,1, 1,1,1,1, L dyes 
h) 0,0, 0, 1, 2, 3, 4, ... 

4. Find a closed form for the generating function for each of 
these sequences. (Assume a general form for the terms of 
the sequence, using the most obvious choice of such a se- 
quence.) 


a) -1,-1, —1, —1, —1, —1, —1, 0, 0, 0, 0, 0, 0, ... 
b) 1,3, 9, 27, 81, 243, 729, ... 

c) 0,0, 3, —3, 3, -3, 3, -3, ... 

D 12,11 D1, Ve Tl, 


7 IN ot 7 af 7 
2 2° aaya saa 
o (2).2(?).24(2).n2?(?), 0.0.00 


f) —3, 3, —3, 3, —-3; 35.<.: 
g) 0,1, —2, 4, —8, 16, —32, 64, ... 
h) 1,0, 1,0, 1,0, 1,0,... 
5. Find a closed form for the generating function for the se- 
quence {a,,}, where 
a) a, =5 for all n =0,1,2,.... 
b) a, = 3” for all n = 0, 1, 2,.... 
c) a, = 2 forn = 3,4, 5, ... and ay = a, =a, = 0. 
d) a, =2n +3 for all n =0,1,2,.... 


e) a, = a for all n = 0, 1,2,.... 


n 


f) a, = net) for all n =0,1,2,.... 
n 


6. Find a closed form for the generating function for the se- 


quence {a,,}, where 


a) a, = —1 for all n =0,1,2,.... 

b) a, = 2” forn = 1, 2, 3, 4, ... and ag = 0. 
c) a, =n—1forn=0,1,2,.... 

d) a, =1/(n+ 1)! for n =0,1,2,.... 


ll 


e) a, = r) forn =0,1,2,.... 
f) a, = 10 ) torn = 01,2). 
n+1 


7. For each of these generating functions, provide a closed 


formula for the sequence it determines. 


a) (x —4) b) +1) 
c) 1/(1 = 5x) d) «°/(1 + 3x) 
e) x +3x+7 + (1/0 —2’)) 


f) 4/0 -x-en 
g) 2/0 -2 h) 2e% 


8. For each of these generating functions, provide a closed 


formula for the sequence it determines. 


a) (+1) b) Bx- 1) 

c) 1/0 — 2x) d) 2/0 — x} 

e) x-1 + (1/0 — 3x)) f) (d +x)/0 +x 
*g) x/(1+x+x?) h) e* —1 


9. Find the coefficient of x!° in the power series of each of 


these functions. 

a) (L+x +x! 4x5 4-9 

b) Gi txt tx tro ta? H 

e) O44 42908 tat 40 42% 407970 tat? t 
B+ xg) 

d) 7? Hat xE 438 +) $26 429 + OTF 
x8 4x1? + o) 

e) (14x72 tatt x6 $84 4x44 84x? 4 + 
x6 4x1? 4 x18 + o) 


10. Find the coefficient of x° in the power series of each of 


these functions. 

a) (1+ x42 499 ++) 

b) G2 + HA + 42° H 

©) 42° HEE HANE 423 tat ee) 
d) (txt Hx Hx 4)? tat t xO $8 + +) 
e) I +x4+2x°) 
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11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


Find the coefficient of x!° in the power series of each of 
these functions. 
a) 1/(1 — 2x) 
© 1/0 -x7 
e) x*/(1 -3x 
Find the coefficient of x!? in the power series of each of 
these functions. 
a) 1/(1 + 3x) 
© 1/0 +58 
e) 2/0 + 4x)? 
Use generating functions to determine the number of dif- 
ferent ways 10 identical balloons can be given to four 
children if each child receives at least two balloons. 


b) 1/0 +x 
d) 1/(1 + 2x) 


b) 1/0 — 2x) 
d) 1/0 — 4x} 


Use generating functions to determine the number of dif- 
ferent ways 12 identical action figures can be given to five 
children so that each child receives at most three action 
figures. 

Use generating functions to determine the number of dif- 
ferent ways 15 identical stuffed animals can be given to 
six children so that each child receives at least one but no 
more than three stuffed animals. 


Use generating functions to find the number of ways to 

choose a dozen bagels from three varieties—egg, salty, 

and plain—if at least two bagels of each kind but no more 
than three salty bagels are chosen. 

In how many ways can 25 identical donuts be distributed 

to four police officers so that each officer gets at least 

three but no more than seven donuts? 

Use generating functions to find the number of ways to 

select 14 balls from a jar containing 100 red balls, 100 

blue balls, and 100 green balls so that no fewer than 3 

and no more than 10 blue balls are selected. Assume that 

the order in which the balls are drawn does not matter. 

What is the generating function for the sequence {c,}, 

where c, is the number of ways to make change for k dol- 

lars using $1 bills, $2 bills, $5 bills, and $10 bills? 

What is the generating function for the sequence {c,}, 

where c, represents the number of ways to make change 

for k pesos using bills worth 10 pesos, 20 pesos, 50 pesos, 
and 100 pesos? 

Give a combinatorial interpretation of the coefficient 

of x* in the expansion (1 +x +27? +x + -+-)°. Use this 

interpretation to find this number. 

Give a combinatorial interpretation of the coefficient 

of xê in the expansion (1 txt 4x0 +)". Use this 

interpretation to find this number. 

a) What is the generating function for {a,}, where a, 
is the number of solutions of x, + x, + x3 = k when 
Xp X2, and x; are integers with x, > 2,0 < x) < 3, and 
2<%, <5? 

b) Use your answer to part (a) to find ag. 

a) What is the generating function for {a,}, where a, 
is the number of solutions of x, +x, + x3 + x4 = k 
when x4, X2, X3, and x, are integers with x, > 3, 1 < 
X < 5,0 < x, <4, and x, > 1? 

b) Use your answer to part (a) to find a}. 


25. 


26. 


27. 


28. 


29. 


Explain how generating functions can be used to find the 
number of ways in which postage of r cents can be pasted 
on an envelope using 3-cent, 4-cent, and 20-cent stamps. 


a) Assume that the order the stamps are pasted on does 
not matter. 

b) Assume that the order in which the stamps are pasted 
on matters. 

c) Use your answer to part (a) to determine the number 
of ways 46 cents of postage can be pasted on an enve- 
lope using 3-cent, 4-cent, and 20-cent stamps when 
the order the stamps are pasted on does not matter. 


(Use of a computer algebra program is advised.) 
d) Use your answer to part (b) to determine the number 


of ways 46 cents of postage can be pasted in a row on 
an envelope using 3-cent, 4-cent, and 20-cent stamps 
when the order in which the stamps are pasted on mat- 
ters. (Use of a computer algebra program is advised.) 


Explain how generating functions can be used to find the 

number of ways in which postage of r cents can be pasted 

on an envelope using 2-cent, 7-cent, 13-cent, and 32-cent 
stamps. 

a) Assume that the order the stamps are pasted on does 
not matter. 

b) Assume that the order the stamps are pasted on mat- 
ters. 

c) Use your answer to part (a) to determine the number 
of ways 49 cents of postage can be pasted on an enve- 
lope using 2-cent, 7-cent, 13-cent, and 32-cent stamps 
when the order the stamps are pasted on does not mat- 
ter. (Use of a computer algebra program is advised.) 

d) Use your answer to part (b) to determine the number of 
ways 49 cents of postage can be pasted on an envelope 
using 2-cent, 7-cent, 13-cent, and 32-cent stamps when 
the order in which the stamps are pasted on matters. (Use 
of a computer algebra program is advised.) 

Customers at a quirky tropical fruit stand can buy at most 

four mangos, at most two passion fruit, any even number 

of papayas, three or more coconuts, and carambolas in 
groups of five. 

a) Explain how generating functions can be used to find 
the number of ways a customer can buy n pieces of 


these fruits, following the restrictions listed. 
b) Use your answer in part (a) to determine the number 


of ways you can buy a dozen pieces of these fruits. 
a) Show that 1/(1 2 _ x3 — x4 — 3° — x) is the 


Xx nw — x x 
generating function for the number of ways that the 
sum 7 can be obtained when a die is rolled repeatedly 
and the order of the rolls matters. 

b) Use part (a) to find the number of ways to roll a total 
of 8 when a die is rolled repeatedly, and the order of 
the rolls matters. (Use of a computer algebra package 
is advised.) 

Use generating functions (and a computer algebra pack- 

age, if available) to find the number of ways to make 

change for $1 using 

a) dimes and quarters. 

b) nickels, dimes, and quarters. 

c) pennies, dimes, and quarters. 

d) pennies, nickels, dimes, and quarters. 


30. Use generating functions (and a computer algebra pack- 
age, if available) to find the number of ways to make 
change for $1 using pennies, nickels, dimes, and quarters 
with 
a) no more than 10 pennies. 

b) no more than 10 pennies and no more than 10 nickels. 
*¢) no more than 10 coins. 

31. Use generating functions to find the number of ways to 
make change for $100 using 
a) $10, $20, and $50 bills. 

b) $5, $10, $20, and $50 bills. 

c) $5, $10, $20, and $50 bills if at least one bill of each 
denomination is used. 

d) $5, $10, and $20 bills if at least one and no more than 
four of each denomination is used. 


32. If G(x) is the generating function for the sequence {a,}, 
what is the generating function for each of these se- 
quences? 

a) 2d, 2d), 2a}, 2d3, ... 

b) 0, dp, a), a2, a3, ... (assuming that terms follow the 
pattern of all but the first term) 

c) 0, 0, 0, 0, ay, a3, ... (assuming that terms follow the 
pattern of all but the first four terms) 

d) dy, d3, dy, .-. 

e) a), 2dy, 343, 4a4, ... [Hint: Calculus required here.] 

f) as 2agaā;, a; + 2d, 2aga; +2aja,, 2aga4 + 
2a, a3 +a, wes 

33. If G(x) is the generating function for the sequence {a,}, 
what is the generating function for each of these se- 
quences? 


a) 0,0, 0, a3, a4, ds, ... (assuming that terms follow the 
pattern of all but the first three terms) 

b) do, 0, a, 0, ay, 0, ... 

c) 0, 0, 0, 0, dp, a), do, ... (assuming that terms follow 
the pattern of all but the first four terms) 

d) do, 2a,, 4a, 8a3, 16a}, ... 

e) 0, ap, 4,/2, ay/3, a;/4, ... 
here. | 

f) ap, dg +4), dg +4; +5, dy + a, +a, +43, ... 


[Hint: Calculus required 


34. Use generating functions to solve the recurrence relation 

a, = Ta,_, with the initial condition ay = 5. 

35. Use generating functions to solve the recurrence relation 

dy, = 3a,_,; + 2 with the initial condition aj = 1. 

36. Use generating functions to solve the recurrence relation 
a, = 3a,_, + 4! with the initial condition ay = 1. 

37. Use generating functions to solve the recurrence rela- 

tion a, = 5a,_, — 6a,_ with initial conditions ay = 6 

and a, = 30. 

38. Use generating functions to solve the recurrence relation 

a, = Ag_ + 2a, + 2* with initial conditions ay = 4 and 


a, = 12. 

39. Use generating functions to solve the recurrence rela- 
tion a, = 4a,_; — 4a,_, + k? with initial conditions ay = 
2 anda, = 5. 


40. Use generating functions to solve the recurrence rela- 
tion a, = 2a,_; + 3a,_y + 4* + 6 with initial conditions 
dy = 20, a, = 60. 
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41. Use generating functions to find an explicit formula for 
the Fibonacci numbers. 


* 42. a) Show that if n is a positive integer, then 


TROIS 


b) Use the extended binomial theorem and part (a) to 
show that the coefficient of x” in the expansion of 
(1 — 4x)7!/? is (°") for all nonnegative integers n. 

* 43. (Calculus required ) Let {C,,} be the sequence of Catalan 
numbers, that is, the solution to the recurrence relation 
C=). : o CkCn-x-1 With Cy = C, = 1 (see Example 5 
in Section 8. 1); 


a) Show that if G(x) is the generating function for the 
sequence of Catalan numbers, then xG(x)? — G(x) + 
1 =0. Conclude (using the initial conditions) that 


G(x) = (1 — y 1 — 4x)/(2x). 


b) Use Exercise 42 to conclude that 


z 1 2n 
co $ L (2) an 
@) pees, n . 


so that 


ge 1 ‘ear 
n+1\n 


c) Show that C, > 2”"! for all positive integers n. 


44. Use generating functions to prove Pascal’s identity: 
C(a, r) = C(n — 1, r) + C(n — 1, r— 1) when n and r are 
positive integers with r < n. [Hint: Use the identity (1 + 
xy = (14x)! 4x0 4+4)"1] 


45. Use generating functions to prove Vandermonde’s iden- 
tity: C(m+n,r) = $}, o COn, r—k)C(n,k), whenever 
m, n, and r are nonnegative integers with r not exceed- 
ing either m or n. [Hint: Look at the coefficient of x” in 
both sides of (1 +.xy"*" = (1+ x)"(1 +x)".] 


46. This exercise shows how to use generating functions to 
derive a formula for the sum of the first n squares. 
a) Show that (x?+x)/(1—x)* is the generat- 
ing function for the sequence {a,}, where 
a, = 12 +2 4-477. 
b) Use part (a) to find an explicit formula for the sum 
P2 + +n. 
The exponential generating function for the sequence {a,,} 
is the series 


co 
La 
ai nn! 


For example, the exponential generating function for the 
sequence 1, 1, 1,... is the function Èr ox/n! =e, 
(You will find this particular series useful in these exercises.) 
Note that e* is the (ordinary) generating function for the 
sequence 1, 1, 1/2!,1/3!, 1/4!,.... 
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47. Find a closed form for the exponential generating func- 
tion for the sequence {a,,}, where 

a) a, =2. b) a, =(-1)". 

c) a, = 3”. d) a,=n+1. 

e) a, =1/(n+1). 

48. Find a closed form for the exponential generating func- 
tion for the sequence {a,,}, where 

a) a, = (-2)”. b) a, =-l1. 

c) a, =n. d) a, = n(n- 1). 

e) a, =1/((n+ 1)\(n +2). 

49. Find the sequence with each of these functions as its ex- 
ponential generating function. 

a) f(x) =e“ b) f(x) = 3x7" 

c) f(x) = e* — 3e” d) f@)=(1-xy) +e 

e) f(x) =e — (1/1 —») 

f) fœ =e**— (1+) + (1/1 — 20) 

g) f(x) =e 

50. Find the sequence with each of these functions as its ex- 
ponential generating function. 

a) f(x) =e" b) f(@) = 2e 

o) f@=e*+e* d) f(x) = (1+ 2x) + e 

e) f(x) =e - (1/0 +) 

£) f(x) = xe" g) JO) = e 

51. A coding system encodes messages using strings of oc- 
tal (base 8) digits. A codeword is considered valid if and 
only if it contains an even number of 7s. 

a) Find a linear nonhomogeneous recurrence relation for 
the number of valid codewords of length n. What are 
the initial conditions? 

b) Solve this recurrence relation using Theorem 6 in 
Section 8.2. 

c) Solve this recurrence relation using generating func- 
tions. 

*52. A coding system encodes messages using strings of 
base 4 digits (that is, digits from the set {0, 1, 2,3}). A 
codeword is valid if and only if it contains an even num- 
ber of Os and an even number of Is. Let a„ equal the 
number of valid codewords of length n. Furthermore, 
let b„, C, and d, equal the number of strings of base 4 dig- 


wn 


its of length n with an even number of Os and an odd num- Links » 


ber of 1s, with an odd number of Os and an even number 

of 1s, and with an odd number of Os and an odd number 

of 1s, respectively. 

a) Show that d, = 4" —a, — b, —c,. Use this to show 
that a,,, = 2a, +b, + Cp bng =b, — Ca +4", and 
Cay] = Cy, b, +4". 

b) What are a, b,, c}, and d,? 

c) Use parts (a) and (b) to find a3, b3, c3, and d}. 

d) Use the recurrence relations in part (a), together with 
the initial conditions in part (b), to set up three equa- 
tions relating the generating functions A(x), B(x), and 
C(x) for the sequences {a,,}, {b,,}, and {c,,}, respec- 
tively. 

e) Solve the system of equations from part (d) to get ex- 
plicit formulae for A(x), B(x), and C(x) and use these 
to get explicit formulae for a,, b,, c,, and d,,. 


n’ n? 


n’ 


n? 


Generating functions are useful in studying the number of 

different types of partitions of an integer n. A partition of 

a positive integer is a way to write this integer as the sum 

of positive integers where repetition is allowed and the or- 

der of the integers in the sum does not matter. For exam- 

ple, the partitions of 5 (with no restrictions) are 1 + 1 +1 + 

1+1,14+14+142,14+14+3,14+24+2,14+4,2+3,and5. 

Exercises 53-58 illustrate some of these uses. 

53. Show that the coefficient p(n) of x” in the formal 
power series expansion of 1/((1—x)(1—x7)(1—x3) +) 
equals the number of partitions of n. 

54. Show that the coefficient p,(m) of x" in the formal 
power series expansion of 1/((1—x)(1—x°)(1—2°) --) 
equals the number of partitions of n into odd integers, that 
is, the number of ways to write n as the sum of odd posi- 
tive integers, where the order does not matter and repeti- 
tions are allowed. 

55. Show that the coefficient p(n) of x” in the formal power 
series expansion of (1 +.x)(1 +.x7)(1 +x°) +- equals the 
number of partitions of n into distinct parts, that is, the 
number of ways to write n as the sum of positive inte- 
gers, where the order does not matter but no repetitions 
are allowed. 

56. Find p,(n), the number of partitions of n into odd parts 
with repetitions allowed, and p,(m), the number of par- 
titions of n into distinct parts, for 1 <n < 8, by writing 
each partition of each type for each integer. 

57. Show that if n is a positive integer, then the number of 
partitions of into distinct parts equals the number 
of partitions of n into odd parts with repetitions allowed; 
that is, p,(n) = p,(n). [Hint: Show that the generating 
functions for p,(n) and p,(n) are equal.] 


** 58. (Requires calculus) Use the generating function of p(n) 


to show that p(n) < eCV" for some constant C. [Hardy 
and Ramanujan showed that p(n) ~ et V2/3yn / (4/3n), 
which means that the ratio of p(n) and the right-hand side 
approaches 1 as n approaches infinity. ] 
Suppose that X is a random variable on a sample space S such 
that X(s) is a nonnegative integer for all s € S. The probabil- 
ity generating function for X is 


Gx(x) = DI p(X(s) = D. 
k=0 


59. (Requires calculus) Show that if Gy is the probability 
generating function for a random variable X such that 
X(s) is a nonnegative integer for all s € S, then 
a) Gy) =1. b) EX) = GC). 
© V(X) = GJ) + Gd) - Gi(1). 

60. Let X be the random variable whose value is n if the 
first success occurs on the nth trial when independent 
Bernoulli trials are performed, each with probability of 
success p. 

a) Find a closed formula for the probability generating 
function Gy. 

b) Find the expected value and the variance of X using 
Exercise 59 and the closed form for the probability 
generating function found in part (a). 
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61. Let m be a positive integer. Let X,, be the random vari- b) Find the expected value and the variance of X,,, using 
able whose value is n if the mth success occurs on the Exercise 59 and the closed form for the probability 

(n + m)th trial when independent Bernoulli trials are per- generating function in part (a). 
formed, each with probability of success p. 62. Show that if X and Y are independent random variables 
a) Using Exercise 32 in the Supplementary Exercises of on a sample space S such that X(s) and Y(s) are nonneg- 
Chapter 7, show that the probability generating func- ative integers for all s € S, then Gy, y(x) = Gy(x)Gy(x). 


tion Gy is given by Gy (x) = p”/(1 — qx)", where 


q=1-p. 


Inclusion—Exclusion 


EXAMPLE 1 


8.5.1 Introduction 


A discrete mathematics class contains 30 women and 50 sophomores. How many students in 
the class are either women or sophomores? This question cannot be answered unless more in- 
formation is provided. Adding the number of women in the class and the number of sophomores 
probably does not give the correct answer, because women sophomores are counted twice. This 
observation shows that the number of students in the class that are either sophomores or women 
is the sum of the number of women and the number of sophomores in the class minus the num- 
ber of women sophomores. A technique for solving such counting problems was introduced 
in Section 6.1. In this section we will generalize the ideas introduced in that section to solve 
problems that require us to count the number of elements in the union of more than two sets. 


8.5.2 The Principle of Inclusion—Exclusion 


How many elements are in the union of two finite sets? In Section 2.2 we showed that the number 
of elements in the union of the two sets A and B is the sum of the numbers of elements in the 
sets minus the number of elements in their intersection. That is, 


JAU B| = JA] + |B] -= |A n Bl. 


As we showed in Section 6.1, the formula for the number of elements in the union of two sets 
is useful in counting problems. Examples 1-3 provide additional illustrations of the usefulness 
of this formula. 


In a discrete mathematics class every student is a major in computer science or mathematics, or 
both. The number of students having computer science as a major (possibly along with math- 
ematics) is 25; the number of students having mathematics as a major (possibly along with 
computer science) is 13; and the number of students majoring in both computer science and 
mathematics is 8. How many students are in this class? 


Solution: Let A be the set of students in the class majoring in computer science and B be the set 
of students in the class majoring in mathematics. Then A N B is the set of students in the class 
who are joint mathematics and computer science majors. Because every student in the class 
is majoring in either computer science or mathematics (or both), it follows that the number of 
students in the class is |A U B|. Therefore, 


|A U B| = |A| + |B] — JA A B| 
= 25 + 13 - 8 = 30. 


Therefore, there are 30 students in the class. This computation is illustrated in Figure 1. < 
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|A UB| = |A| + |B|- |A n B| = 254+ 13-8 =30 |A U B| = |A| + |B| - |A N B| = 142 + 90 -12 = 220 


|A| =25 |An B| =8 |B| = 13 JA] = 142 |AN Bl =12 |B| =90 
FIGURE 1 The set of students in a FIGURE 2 The set of positive integers not 
discrete mathematics class. exceeding 1000 divisible by either 7 or 11. 
EXAMPLE 2 How many positive integers not exceeding 1000 are divisible by 7 or 11? 


EXAMPLE 3 


Solution: Let A be the set of positive integers not exceeding 1000 that are divisible by 7, and 
let B be the set of positive integers not exceeding 1000 that are divisible by 11. Then AUB 
is the set of integers not exceeding 1000 that are divisible by either 7 or 11, and A N B is the 
set of integers not exceeding 1000 that are divisible by both 7 and 11. From Example 2 of 
Section 4.1, we know that among the positive integers not exceeding 1000 there are [1000/7] 
integers divisible by 7 and [1000/11] divisible by 11. Because 7 and 11 are relatively prime, 
the integers divisible by both 7 and 11 are those divisible by 7 - 11. Consequently, there are 
[1000/(11-7)] positive integers not exceeding 1000 that are divisible by both 7 and 11. It 
follows that there are 


JA U B| = |A| + |B| -|AnB| 
= | 1000), 2] 
“L7 11 7-11 
= 142 +90 — 12 = 220 


positive integers not exceeding 1000 that are divisible by either 7 or 11. This computation is 
illustrated in Figure 2. < 


Example 3 shows how to find the number of elements in a finite universal set that are outside 
the union of two sets. 


Suppose that there are 1807 freshmen at your school. Of these, 453 are taking a course in 
computer science, 567 are taking a course in mathematics, and 299 are taking courses in both 
computer science and mathematics. How many are not taking a course either in computer sci- 
ence or in mathematics? 


Solution: To find the number of freshmen who are not taking a course in either mathematics 
or computer science, subtract the number that are taking a course in either of these subjects 
from the total number of freshmen. Let A be the set of all freshmen taking a course in com- 
puter science, and let B be the set of all freshmen taking a course in mathematics. It follows 
that |A| = 453, |B| = 567, and |A N B| = 299. The number of freshmen taking a course in ei- 
ther computer science or mathematics is 


JA U B| = |A| + |B] — |A N B| = 453 + 567 — 299 = 721. 
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(a) Count of elements by (b) Count of elements by (c) Count of elements by 
lal + lB] + [cl |a] + lB] + |c] -|A n B| — la] + lB] + [Cc] - |A n Bl — 
|JAnc|—|Bnc| |JAnc|—|Bnc|+|AnBnc| 


FIGURE 3 Finding a formula for the number of elements in the union of three sets. 


EXAMPLE 4 


Consequently, there are 1807 — 721 = 1086 freshmen who are not taking a course in computer 
science or mathematics. < 


We will now begin our development of a formula for the number of elements in the union 
of a finite number of sets. The formula we will develop is called the principle of inclusion- 
exclusion. For concreteness, before we consider unions of n sets, where n is any positive integer, 
we will derive a formula for the number of elements in the union of three sets A, B, and C. To 
construct this formula, we note that |A| + |B] + [C| counts each element that is in exactly one 
of the three sets once, elements that are in exactly two of the sets twice, and elements in all three 
sets three times. This is illustrated in the first panel in Figure 3. 

To remove the overcount of elements in more than one of the sets, we subtract the number 
of elements in the intersections of all pairs of the three sets. We obtain 


|A| + |B] + |C] — |AN B] — |ANC|-|BnC}. 


This expression still counts elements that occur in exactly one of the sets once. An element 
that occurs in exactly two of the sets is also counted exactly once, because this element will 
occur in one of the three intersections of sets taken two at a time. However, those elements 
that occur in all three sets will be counted zero times by this expression, because they occur 
in all three intersections of sets taken two at a time. This is illustrated in the second panel in 
Figure 3. 

To remedy this undercount, we add the number of elements in the intersection of all three 
sets. This final expression counts each element once, whether it is in one, two, or three of the 
sets. Thus, 


|AUBUC| =|A| + |B] +|C] -|ANB|-|ANC|—-|BNC]+|AnBn Cl. 


This formula is illustrated in the third panel of Figure 3. 
Example 4 illustrates how this formula can be used. 


A total of 1232 students have taken a course in Spanish, 879 have taken a course in French, 
and 114 have taken a course in Russian. Further, 103 have taken courses in both Spanish and 
French, 23 have taken courses in both Spanish and Russian, and 14 have taken courses in both 
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ISA FOR| =? [SN F| = 103 


|S R| = 23 


|R| = 114 
|SU FU R| = 2092 


FIGURE 4 The set of students who have taken courses 
in Spanish, French, and Russian. 


French and Russian. If 2092 students have taken at least one of Spanish, French, and Russian, 
how many students have taken a course in all three languages? 


Solution: Let S be the set of students who have taken a course in Spanish, F the set of students 
who have taken a course in French, and R the set of students who have taken a course in Russian. 
Then 


|S] = 1232, |F| = 879, |R| = 114, 

IS n F| = 103, |S N R| = 23, |F A R| = 14, 
and 

IS U F U R| = 2092. 
When we insert these quantities into the equation 

ISU FUR] = |S| + |F| + IRI- ISnF|-ISAR|-|FAaRI+ISAFAR|İ 
we obtain 

2092 = 1232 + 879 + 114 — 103 — 23 —- 14+ |SN FNRI. 


We now solve for [SQA FOR. We find that |SAF AR| =7. Therefore, there are seven 
students who have taken courses in Spanish, French, and Russian. This is illustrated in 
Figure 4. < 


We will now state and prove the inclusion—exclusion principle for n sets, where n is a 
positive integer. This priniciple tells us that we can count the elements in a union of n sets by 
adding the number of elements in the sets, then subtracting the sum of the number of elements 
in all intersections of two of these sets, then adding the number of elements in all intersections 


THEOREM 1 


EXAMPLE 5 


Extra 
Examples 


» 
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of three of these sets, and so on, until we reach the number of elements in the intersection of 
all the sets. It is added when there is an odd number of sets and added when there is an even 
number of sets. 


THE PRINCIPLE OF INCLUSION-EXCLUSION Let A,,A,,...,A, be finite sets. 
Then 


|4, UA, U = UA, = È, Mil- $, MNA; 


1l<i<n l<i<j<n 
+ È [AN Ayn Ay = HODHA NANA NAY: 
1<i<j<k<n 


Proof: We will prove the formula by showing that an element in the union is counted exactly 
once by the right-hand side of the equation. Suppose that a is a member of exactly r of the sets 
A, A5,...,A, where 1 <r <n. This element is counted C(y, 1) times by 2|A,]. It is counted 
C(r, 2) times by X|A;N Aj]. In general, it is counted C(r, m) times by the summation involving 
m of the sets A,. Thus, this element is counted exactly 


C(r, 1) — C(r, 2) + Cy 3) — = + (DCC, r) 


times by the expression on the right-hand side of this equation. Our goal is to evaluate this 
quantity. By Corollary 2 of Section 6.4, we have 


C(x, 0) — C(r, 1) + Cr, 2) — = + (—1)"C(r, r) = 0. 
Hence, 
1 = C(r, 0) = C(r, 1) — Cr, 2) + + + (~1) +! C(s r). 


Therefore, each element in the union is counted exactly once by the expression on the right-hand 
side of the equation. This proves the principle of inclusion—exclusion. < 


The inclusion—exclusion principle gives a formula for the number of elements in the union 
of n sets for every positive integer n. There are terms in this formula for the number of ele- 
ments in the intersection of every nonempty subset of the collection of the n sets. Hence, there 
are 2” — 1 terms in this formula. 


Give a formula for the number of elements in the union of four sets. 


Solution: The inclusion—exclusion principle shows that 


|A, UA, UA; UAg| = |A| + [Ao] + 13] + |441 
— [Ay 042| = |A; N43] — |A; NA; = [Ag AZ] = JA32 N 44| 
— |4; NAg| + |A] NA, N43] + [A] NA, NAg| + |A; N A; NAGI 
+ |A, NA; N A4] — |A; N4; NA; NASI. 


Note that this formula contains 15 different terms, one for each nonempty subset of 
{A,, Ay, A3, Ag}. < 
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Exercises 


. How many elements are in A, UA, if there are 12 ele- 
ments in A,, 18 elements in A, and 

a) A, NA, =Ø? b) [A NA,| = 1? 

c) [4 NA,| = 6? d) A, CA,? 

. There are 345 students at a college who have taken a 
course in calculus, 212 who have taken a course in dis- 
crete mathematics, and 188 who have taken courses in 
both calculus and discrete mathematics. How many stu- 
dents have taken a course in either calculus or discrete 
mathematics? 


. A survey of households in the United States reveals that 
96% have at least one television set, 98% have telephone 
service, and 95% have telephone service and at least 
one television set. What percentage of households in the 
United States have neither telephone service nor a televi- 
sion set? 


. A marketing report concerning personal computers states 
that 650,000 owners will buy a printer for their machines 
next year and 1,250,000 will buy at least one software 
package. If the report states that 1,450,000 owners will 
buy either a printer or at least one software package, how 
many will buy both a printer and at least one software 
package? 


. Find the number of elements in A, UA, UA; if there 
are 100 elements in each set and if 


a) the sets are pairwise disjoint. 


b) there are 50 common elements in each pair of sets and 
no elements in all three sets. 


c) there are 50 common elements in each pair of sets and 
25 elements in all three sets. 


d) the sets are equal. 

. Find the number of elements in A, U A, UA; if there are 
100 elements in A,, 1000 in A,, and 10,000 in A; if 

a) A, CA, and A, CA3. 

b) the sets are pairwise disjoint. 


c) there are two elements common to each pair of sets 
and one element in all three sets. 


. There are 2504 computer science students at a school. Of 
these, 1876 have taken a course in Java, 999 have taken a 
course in Linux, and 345 have taken a course in C. Fur- 
ther, 876 have taken courses in both Java and Linux, 231 
have taken courses in both Linux and C, and 290 have 
taken courses in both Java and C. If 189 of these stu- 
dents have taken courses in Linux, Java, and C, how many 
of these 2504 students have not taken a course in any of 
these three programming languages? 


. Ina survey of 270 college students, it is found that 64 like 
Brussels sprouts, 94 like broccoli, 58 like cauliflower, 
26 like both Brussels sprouts and broccoli, 28 like both 
Brussels sprouts and cauliflower, 22 like both broccoli 


10. 


11. 


12. 


13. 


14. 


15. 


*16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


and cauliflower, and 14 like all three vegetables. How 
many of the 270 students do not like any of these 
vegetables? 


. How many students are enrolled in a course either in cal- 


culus, discrete mathematics, data structures, or program- 
ming languages at a school if there are 507, 292, 312, 
and 344 students in these courses, respectively; 14 in 
both calculus and data structures; 213 in both calculus 
and programming languages; 211 in both discrete math- 
ematics and data structures; 43 in both discrete mathe- 
matics and programming languages; and no student may 
take calculus and discrete mathematics, or data structures 
and programming languages, concurrently? 

Find the number of positive integers not exceeding 100 
that are not divisible by 5 or by 7. 

Find the number of positive integers not exceeding 1000 
that are not divisible by 3, 17, or 35. 

Find the number of positive integers not exceeding 
10,000 that are not divisible by 3, 4, 7, or 11. 

Find the number of positive integers not exceeding 100 
that are either odd or the square of an integer. 

Find the number of positive integers not exceeding 1000 
that are either the square or the cube of an integer. 

How many bit strings of length eight do not contain six 
consecutive 0s? 

How many permutations of the 26 letters of the English 
alphabet do not contain any of the strings fish, rat or bird? 
How many permutations of the 10 digits either begin with 
the 3 digits 987, contain the digits 45 in the fifth and sixth 
positions, or end with the 3 digits 123? 

How many elements are in the union of four sets if 
each of the sets has 100 elements, each pair of the sets 
shares 50 elements, each three of the sets share 25 ele- 
ments, and there are 5 elements in all four sets? 

How many elements are in the union of four sets if the 
sets have 50, 60, 70, and 80 elements, respectively, each 
pair of the sets has 5 elements in common, each triple of 
the sets has 1 common element, and no element is in all 
four sets? 

How many terms are there in the formula for the number 
of elements in the union of 10 sets given by the principle 
of inclusion—exclusion? 

Write out the explicit formula given by the principle of 
inclusion—exclusion for the number of elements in the 
union of five sets. 

How many elements are in the union of five sets if the 
sets contain 10,000 elements each, each pair of sets has 
1000 common elements, each triple of sets has 100 com- 
mon elements, every four of the sets have 10 common 
elements, and there is 1 element in all five sets? 

Write out the explicit formula given by the principle of 
inclusion—exclusion for the number of elements in the 
union of six sets when it is known that no three of these 
sets have a common intersection. 


*24. 


25. 


26. 


27. 


Prove the principle of inclusion—exclusion using mathe- 
matical induction. 

Let E,, E,, and E be three events from a sample space S. 
Find a formula for the probability of E, U E, U E3. 

Find the probability that when a fair coin is flipped five 
times tails comes up exactly three times, the first and last 
flips come up tails, or the second and fourth flips come 
up heads. 

Find the probability that when four numbers from 1 to 
100, inclusive, are picked at random with no repetitions 
allowed, either all are odd, all are divisible by 3, or all are 
divisible by 5. 


28. 


29. 


30. 


31. 
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Find a formula for the probability of the union of four 
events in a sample space if no three of them can occur at 
the same time. 


Find a formula for the probability of the union of five 
events in a sample space if no four of them can occur at 
the same time. 


Find a formula for the probability of the union of n events 
in a sample space when no two of these events can occur 
at the same time. 


Find a formula for the probability of the union of n events 
in a sample space. 


8.6 Applications of Inclusion—Exclusion 


8.6.1 Introduction 


Many counting problems can be solved using the principle of inclusion—exclusion. For instance, 
we can use this principle to find the number of primes less than a positive integer. Many prob- 
lems can be solved by counting the number of onto functions from one finite set to another. The 
inclusion—exclusion principle can be used to find the number of such functions. The well-known 
hatcheck problem can be solved using the principle of inclusion-exclusion. This problem asks 
for the probability that no person is given the correct hat back by a hatcheck person who gives 


the hats back randomly. 


8.6.2 An Alternative Form of Inclusion—Exclusion 


There is an alternative form of the principle of inclusion—exclusion that is useful in counting 
problems. In particular, this form can be used to solve problems that ask for the number of 


elements in a set that have none of n properties P4, P3, ..., P 


n* 


Let A; be the subset containing the elements that have property P;. The number 
of elements with all the properties P;,P;,,...,P; will be denoted by M(P; P, ... P) 
Writing these quantities in terms of sets, we have g 


|A; NA, N+ AA l = NP; P, Pa) 


Tk 


If the number of elements with none of the properties P4, P3, ..., 


P,, is denoted by N(P{ P} ... P!) 


and the number of elements in the set is denoted by N, it follows that 


N(P! P} ... P!) =N- |A UA, U = UA 


al 


From the inclusion—exclusion principle, we see that 


NPP}. P) =N- Y, NP)+ Y NEP) 


l<i<n 


- È NOP PP,) + + (-1)"N(P Pp... Py): 


l<i<j<k<n 


l<i<j<n 


n 
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EXAMPLE 1 


Example 1 shows how the principle of inclusion—exclusion can be used to determine the 
number of solutions in integers of an equation with constraints. 


How many solutions does 
x +X +x; = 11 
have, where x), x, and x3, are nonnegative integers with x, < 3, x) < 4, and x, < 6? 


Solution: To apply the principle of inclusion—-exclusion, let a solution have property P, 
if xı > 3, property P, if x, > 4, and property P, if x; > 6. The number of solutions satisfy- 
ing the inequalities x, < 3, x, < 4, and x; < 6 is 


N(P! PLP!) = N — N(P,) — N(P2) — N(P3) + N(P,P,) 
+ N(P,P3) + N(PP3) — N(P}P>P3). 


Using the same techniques as in Example 5 of Section 6.5, it follows that 


> N = total number of solutions = C(3 + 11 — 1, 11) = 78, 

> N(P,) = (number of solutions with x, > 4) = C(3 +7 — 1,7) = CQ, 7) = 36, 

> N(P,) = (number of solutions with x, > 5) = C(3 + 6 — 1, 6) = C(8, 6) = 28, 

> N(P3) = (number of solutions with x, > 7) = C(3 + 4 — 1,4) = C(6, 4) = 15, 

> N(P,P,) = (number of solutions with x, > 4 and x, > 5) = C(3 +2 — 1, 2) = 
C(4, 2) = 6, 

> N(P P3) = (number of solutions with x, > 4 and x, > 7) = C(3 +0- 1,0)=1, 

> N(P P3) = (number of solutions with x, > 5 and x; > 7) = 0, 

> N(P,P,P3) = (number of solutions with x, > 4, x > 5, and x; > 7) = 0. 


Inserting these quantities into the formula for N(P{ P}P}) shows that the number of solutions 
with x, <3, x, <4, and x; < 6 equals 


N(P' PLP) = 78 — 36-28 -15+6+1+0-0=6. < 


8.6.3 The Sieve of Eratosthenes 


In Section 4.3 we showed how to use the sieve of Eratosthenes to find all primes less than a 
specified positive integer n. Using the principle of inclusion—exclusion, we can find the num- 
ber of primes not exceeding a specified positive integer with the same reasoning as is used in 
the sieve of Eratosthenes. Recall that a composite integer is divisible by a prime not exceeding 
its square root. So, to find the number of primes not exceeding 100, first note that compos- 
ite integers not exceeding 100 must have a prime factor not exceeding 10. Because the only 
primes not exceeding 10 are 2, 3, 5, and 7, the primes not exceeding 100 are these four primes 
and those positive integers greater than 1 and not exceeding 100 that are divisible by none 
of 2, 3, 5, or 7. To apply the principle of inclusion—exclusion, let P, be the property that an 
integer is divisible by 2, let P, be the property that an integer is divisible by 3, let P} be the 
property that an integer is divisible by 5, and let P, be the property that an integer is divisible 
by 7. Thus, the number of primes not exceeding 100 is 


4+N(P\P5P3P,)- 


EXAMPLE 2 
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Because there are 99 positive integers greater than 1 and not exceeding 100, the principle of 
inclusion—exclusion shows that 


N(P! PLPP!) = 99 — N(P,) — N(P2) — N(P3) — N(P4) 
+ N(P;P2) + N(P P3) + NCP) Py) + N(P2P3) + N(P2P4) + N(P3P4) 
—N(P,P,P3) — N(P P,P) — N(P,P3P4) — N(P2P3P.) 
+N(P,P,P3P4). 
The number of integers not exceeding 100 (and greater than 1) that are divisible by all the primes 


in a subset of {2, 3, 5, 7} is [100/N], where N is the product of the primes in this subset. (This 
follows because any two of these primes have no common factor.) Consequently, 


100 100 100 100 
neriPsPs) = 99-[=-|-[>>]-[>-]- [>| 


100 100 100 100 100 100 
7 [55] t [55] t [z] t [55] + [55] + [55] 
-H-2 - H-S] 100 | 
2.3.5 2.3.7 2-5-7 3-5-7 2.3.5.7 


= 99 — 50 — 33 — 20 — 14+ 16+ 10 +7 +6+4+2-3-2-1-0+0 
= 21. 


Hence, there are 4 + 21 = 25 primes not exceeding 100. 


8.6.4 The Number of Onto Functions 


The principle of inclusion—exclusion can also be used to determine the number of onto functions 
from a set with m elements to a set with n elements. First consider Example 2. 


How many onto functions are there from a set with six elements to a set with three elements? 


Solution: Suppose that the elements in the codomain are b,, b}, and b}. Let P,, P}, and P} be 
the properties that b,, b,, and b, are not in the range of the function, respectively. Note that 
a function is onto if and only if it has none of the properties P,, P}, or P}. By the inclusion— 
exclusion principle it follows that the number of onto functions from a set with six elements to 
a set with three elements is 


NCP, PP!) = N — [N(P,) + N(P,) + N(P3)] 
+ [N(P,P,) + N(P|P3) + N(P,P3)] — N(P,P,P3), 


where N is the total number of functions from a set with six elements to one with three elements. 
We will evaluate each of the terms on the right-hand side of this equation. 

From Example 6 of Section 6.1, it follows that N = 3°. Note that N (P;) is the number of 
functions that do not have b; in their range. Hence, there are two choices for the value of the 
function at each element of the domain. Therefore, N(P;) = 2°. Furthermore, there are C(3, 1) 
terms of this kind. Note that N(P;P;) is the number of functions that do not have b, and b; in 
their range. Hence, there is only one choice for the value of the function at each element of 
the domain. Therefore, N(P;P,) = 1° = 1. Furthermore, there are C(3, 2) terms of this kind. 
Also, note that N(P,P,P3) = 0, because this term is the number of functions that have none 
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THEOREM 1 


Counting onto functions 
is much harder than 
counting one-to-one 
functions! 


EXAMPLE 3 


EXAMPLE 4 


of b,,b,, and b, in their range. Clearly, there are no such functions, so the number of onto 
functions from a set with six elements to one with three elements is 


3° — C(3, 1)2° + C(3, 2)1° = 729 — 192 + 3 = 540. < 


The general result that tells us how many onto functions there are from a set with m elements 
to one with n elements will now be stated. The proof of this result is left as an exercise for the 
reader. 


Let m and n be positive integers with m > n. Then, there are 
i = Oo D= 1) CG, a =N = EN Con = o 


onto functions from a set with m elements to a set with n elements. 


An onto function from a set with m elements to a set with n elements corresponds to a way 
to distribute the m elements in the domain to n indistinguishable boxes so that no box is empty, 
and then to associate each of the n elements of the codomain to a box. This means that the 
number of onto functions from a set with m elements to a set with n elements is the number 
of ways to distribute m distinguishable objects to n indistinguishable boxes so that no box is 
empty multiplied by the number of permutations of a set with n elements. Consequently, the 
number of onto functions from a set with m elements to a set with n elements equals n!S(m, n), 
where S(m, n) is a Stirling number of the second kind defined in Section 6.5. This means that 
we can use Theorem 1 to deduce the formula given in Section 6.5 for S(m, n). (See Chapter 6 of 
[MiRo91] for more details about Stirling numbers of the second kind.) 

One of the many different applications of Theorem 1 will now be described. 


How many ways are there to assign five different jobs to four different employees if every em- 
ployee is assigned at least one job? 


Solution: Consider the assignment of jobs as a function from the set of five jobs to the set of 
four employees. An assignment where every employee gets at least one job is the same as an 
onto function from the set of jobs to the set of employees. Hence, by Theorem 1 it follows that 
there are 


45 — C(4, 1)3° + C(4, 2)2° — C(4, 3)1° = 1024 — 972 + 192 — 4 = 240 


ways to assign the jobs so that each employee is assigned at least one job. < 


8.6.5 Derangements 


The principle of inclusion—exclusion will be used to count the permutations of n objects that 
leave no objects in their original positions. Consider Example 4. 


The Hatcheck Problem A new employee checks the hats of n people at a restaurant, forgetting 
to put claim check numbers on the hats. When customers return for their hats, the checker gives 
them back hats chosen at random from the remaining hats. What is the probability that no one 
receives the correct hat? < 


Links > 


EXAMPLE 5 


THEOREM 2 


Links » 
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Remark: The answer is the number of ways the hats can be arranged so that there is no hat in 
its original position divided by n!, the number of permutations of n hats. We will return to this 
example after we find the number of permutations of n objects that leave no objects in their 
original position. 


A derangement is a permutation of objects that leaves no object in its original position. To 
solve the problem posed in Example 4 we will need to determine the number of derangements 
of a set of n objects. 


The permutation 21453 is a derangement of 12345 because no number is left in its original 
position. However, 21543 is not a derangement of 12345, because this permutation leaves 4 
fixed. < 


Let D, denote the number of derangements of n objects. For instance, D} = 2, because the 
derangements of 123 are 231 and 312. We will evaluate D,, for all positive integers n, using the 
principle of inclusion—exclusion. 


The number of derangements of a set with n elements is 


lem Ma 
D, =n [1-245 zt tC =|: 


Proof: Let a permutation have property P; if it fixes element i. The number of derangements is 
the number of permutations having none of the properties P; for i = 1, 2, ..., n. This means that 


Dp = N(P Pa P). 
Using the principle of inclusion—exclusion, it follows that 


D, =N- È NP) + È NPP) - YY NP,PP) + + (-D"N(P {Pp Py) 


i<j i<j<k 


where N is the number of permutations of n elements. This equation states that the number of 
permutations that fix no elements equals the total number of permutations, less the number that 
fix at least one element, plus the number that fix at least two elements, less the number that fix 
at least three elements, and so on. All the quantities that occur on the right-hand side of this 
equation will now be found. 


First, note that N = n!, because N is simply the total number of permutations of n elements. 
Also, N(P;) = (n — 1)!. This follows from the product rule, because N(P;) is the number of 
permutations that fix element i, so the ith position of the permutation is determined, but each 
of the remaining positions can be filled arbitrarily. Similarly, 


NPP) =(n—-2)}, 


HISTORICAL NOTE In rencontres (matches), an old French card game, the 52 cards in a deck are laid out 
in a row. The cards of a second deck are laid out with one card of the second deck on top of each card of 
the first deck. The score is determined by counting the number of matching cards in the two decks. In 1708 
Pierre Raymond de Montmort (1678-1719) posed le problème de rencontres: What is the probability that no 
matches take place in the game of rencontres? The solution to Montmort’s problem is the probability that a 
randomly selected permutation of 52 objects is a derangement, namely, D,,/52!, which, as we will see, is 
approximately 1/e. 
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because this is the number of permutations that fix elements i and j, but where the 
other n — 2 elements can be arranged arbitrarily. In general, note that 


N(P; P, PP, ) = (n- m)!, 


ipo in ** 


because this is the number of permutations that fix elements j,,i,,...,i,,, but where the 
other n — m elements can be arranged arbitrarily. Because there are C(n, m) ways to choose 
m elements from n, it follows that 


> NEP) = Co, Dn- 1)! 


l<i<n 


5 N(P;P) = Cn, 2)(n — 2)!, 


l<i<j<n 


and in general, 


> N(P;, P;, -~ P; ) = C(n, m\(n = m)!. 
1<i, <i, <-++<i,,<n j 


m> 


Consequently, inserting these quantities into our formula for D,, gives 


D, =n! — C(n, 1X(n — 1)! + C@, 2)(n — 2)! — + + C12)" C@, n(n — n)! 


s a he ei 
11m- 1)! 7 2(n— 2)! i mor 


Simplifying this expression gives 


Dan [I-54 5-6 EDE]. 
na nl! r 


It is now straightforward to find D, for a given positive integer n. For instance, using 
Theorem 2, it follows that 


E E ti 
= 3! = pane E = =a am 
D stji I 2 zil 6(1 eG J 3, 


as we have previously remarked. 
The solution of the problem in Example 4 can now be given. 


Solution: The probability that no one receives the correct hat is D,,/n!. By Theorem 2, this 
probability is 


The values of this probability for 2 < n < 7 are displayed in Table 1. 


TABLE 1 The Probability of a Derangement. 


0.50000 0.33333 0.37500 0.36667 0.36806 0.36786 
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By the identity e* = Do x /j! for all real numbers x (from calculus), we know that 
=l = — 1 1 — se. — n 1 oe J 
e =l mta +( DT x 0.368. 


Because this is an alternating series with terms tending to zero, it follows that as n grows without 
bound, the probability that no one receives the correct hat converges to e~! ~ 0.368. In fact, this 
probability can be shown to be within 1/(n + 1)! of e7!. < 


Exercises 


10. 


11. 


12. 


. Suppose that in a bushel of 100 apples there are 20 that 


have worms in them and 15 that have bruises. Only those 
apples with neither worms nor bruises can be sold. If 
there are 10 bruised apples that have worms in them, how 
many of the 100 apples can be sold? 


. Of 1000 applicants for a mountain-climbing trip in the 


Himalayas, 450 get altitude sickness, 622 are not in good 
enough shape, and 30 have allergies. An applicant quali- 
fies if and only if this applicant does not get altitude sick- 
ness, is in good shape, and does not have allergies. If there 
are 111 applicants who get altitude sickness and are not 
in good enough shape, 14 who get altitude sickness and 
have allergies, 18 who are not in good enough shape and 
have allergies, and 9 who get altitude sickness, are not in 
good enough shape, and have allergies, how many appli- 
cants qualify? 


. How many solutions does the equation x, + x, + x3 = 13 


have where x,, x), and x, are nonnegative integers less 
than 6? 


. Find the number of solutions of the equation x, + x, + 


X3 + x4 = 17, where x;, i = 1, 2, 3, 4, are nonnegative in- 
tegers such that x, < 3, x, < 4, x, <5, and x, < 8. 


. Find the number of primes less than 200 using the prin- 


ciple of inclusion—exclusion. 


. An integer is called squarefree if it is not divisible by 


the square of a positive integer greater than 1. Find the 
number of squarefree positive integers less than 100. 


. How many positive integers less than 10,000 are not the 


second or higher power of an integer? 


. How many onto functions are there from a set with seven 


elements to one with five elements? 


. How many ways are there to distribute six different toys 


to three different children such that each child gets at least 
one toy? 


In how many ways can eight distinct balls be distributed 
into three distinct urns if each urn must contain at least 
one ball? 

In how many ways can seven different jobs be assigned 
to four different employees so that each employee is as- 
signed at least one job and the most difficult job is as- 
signed to the best employee? 


List all the derangements of {1, 2, 3, 4}. 


13. 


14. 


15. 


16. 


*17. 


* 18. 


*19. 


20. 
21. 


22. 


*23. 


How many derangements are there of a set with seven 
elements? 

What is the probability that none of 10 people receives 
the correct hat if a hatcheck person hands their hats back 
randomly? 

A machine that inserts letters into envelopes goes hay- 
wire and inserts letters randomly into envelopes. What is 
the probability that in a group of 100 letters 

a) no letter is put into the correct envelope? 

b) exactly one letter is put into the correct envelope? 

c) exactly 98 letters are put into the correct envelopes? 

d) exactly 99 letters are put into the correct envelopes? 

e) all letters are put into the correct envelopes? 

A group of n students is assigned seats for each of two 
classes in the same classroom. How many ways can these 
seats be assigned if no student is assigned the same seat 
for both classes? 

How many ways can the digits 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 be 
arranged so that no even digit is in its original position? 

Use a combinatorial argument to show that the sequence 
{D,,}, where D, denotes the number of derangements 
of n objects, satisfies the recurrence relation 


D, =(n— 1)(D,_1 + D,,_) 


for n > 2. [Hint: Note that there are n — 1 choices for the 
first element k of a derangement. Consider separately the 
derangements that start with k that do and do not have 1 
in the kth position. ] 


Use Exercise 18 to show that 
D,, = nD, + (-1)" 
forn > 1. 
Use Exercise 19 to find an explicit formula for D,,. 


For which positive integers n is D„, the number of de- 


rangements of n objects, even? 


n? 


Suppose that p and q are distinct primes. Use the prin- 
ciple of inclusion—exclusion to find (pq), the number 
of positive integers not exceeding pq that are relatively 
prime to pq. 

Use the principle of inclusion—exclusion to derive a for- 
mula for ġ(n) when the prime factorization of n is 


am 


Spip 
n =p, Py Pm. 
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*24. Show that if n is a positive integer, then 


n! = C(n, 0)D, + Cla, 1I)D,_, 
+- + C(n, n — 1)D; + Cn, n)Do, 


where D, is the number of derangements of k objects. 


Key Terms and Results 


TERMS 


recurrence relation: a formula expressing terms of a se- 
quence, except for some initial terms, as a function of one 
or more previous terms of the sequence 


initial conditions for a recurrence relation: the values of the 
terms of a sequence satisfying the recurrence relation be- 
fore this relation takes effect 


dynamic programming: an algorithmic paradigm that finds 
the solution to an optimization problem by recursively 
breaking down the problem into overlapping subproblems 
and combining their solutions with the help of a recurrence 
relation 


linear homogeneous recurrence relation with constant co- 
efficients: a recurrence relation that expresses the terms of 
a sequence, except initial terms, as a linear combination of 
previous terms 


characteristic roots of a linear homogeneous recurrence re- 
lation with constant coefficients: the roots of the poly- 
nomial associated with a linear homogeneous recurrence 
relation with constant coefficients 


linear nonhomogeneous recurrence relation with constant 
coefficients: a recurrence relation that expresses the terms 
of a sequence, except for initial terms, as a linear combina- 
tion of previous terms plus a function that is not identically 
zero that depends only on the index 


divide-and-conquer algorithm: an algorithm that solves a 
problem recursively by splitting it into a fixed number of 
smaller nonoverlapping subproblems of the same type 


generating function of a sequence: the formal series that has 
the nth term of the sequence as the coefficient of x” 


sieve of Eratosthenes: a procedure for finding the primes less 
than a specified positive integer 


Review Questions 


1. a) What is a recurrence relation? 
b) Find a recurrence relation for the amount of money 
that will be in an account after n years if $1,000,000 
is deposited in an account yielding 9% annually. 


2. Explain how the Fibonacci numbers are used to solve 
Fibonacci’s problem about rabbits. 


3. a) Find a recurrence relation for the number of steps 
needed to solve the Tower of Hanoi puzzle. 


25. How many derangements of {1, 2, 3, 4, 5, 6} begin with 
the integers 1, 2, and 3, in some order? 


26. How many derangements of {1, 2, 3, 4, 5, 6} end with the 
integers 1, 2, and 3, in some order? 


27. Prove Theorem 1. 


derangement: a permutation of objects such that no object is 
in its original place 


RESULTS 


the formula for the number of elements in the union of two 
finite sets: 


|A U B| = |A| + |B| — |A A B| 


the formula for the number of elements in the union of 
three finite sets: 
|JAUBUC| = |A| + |B] +|C] —|AnB| -|ANC| 
-—|BNC|+|ANBnC| 


the principle of inclusion—-exclusion: 


[A4 UAU = UA, = È lAl- È, 14,94) 


l<i<n l<i<j<n 


+ È, JANANA 


l<i<j<k<n 


=- + (CDA NA, N= NA, | 


the number of onto functions from a set with m elements 
to a set with n elements: 


n” — C(n, 1)(n — 1)” + C(n, 2)(n = 2)” 
ees (-1)""!C(, n- 1) E 


the number of derangements of n objects: 


D =n |1- 5+5- 


al 
Ce ey =I 


b) Show how this recurrence relation can be solved using 
iteration. 


4. a) Explain how to find a recurrence relation for the num- 
ber of bit strings of length n not containing two con- 
secutive Is. 


b) Describe another counting problem that has a solution 
satisfying the same recurrence relation. 


10. 


a) What is dynamic programming and how are recur- 
rence relations used in algorithms that follow this 
paradigm? 

b) Explain how dynamic programming can be used to 
schedule talks in a lecture hall from a set of possible 
talks to maximize overall attendance. 


Define a linear homogeneous recurrence relation of de- 

gree k. 

a) Explain how to solve linear homogeneous recurrence 
relations of degree 2. 


b) Solve the recurrence relation a, = 13a,_; — 22a,_5 
forn > 2 if ay = 3 anda, = 15. 
c) Solve the recurrence relation a, = 14a,_; — 49a,_> 


for n > 2 if dg = 3 anda, = 35. 


a) Explain how to find f(b‘) where k is a positive integer 
if f(n) satisfies the divide-and-conquer recurrence re- 
lation f(n) = af (n /b) + g(n) whenever b divides the 
positive integer n. 

b) Findf(256) if f(n) = 3f(n/4) + 5n/4 and f(1) = 7. 

a) Derive a divide-and-conquer recurrence relation for 
the number of comparisons used to find a number in 
a list using a binary search. 

b) Give a big-O estimate for the number of comparisons 
used by a binary search from the divide-and-conquer 
recurrence relation you gave in (a) using Theorem 1 
in Section 8.3. 

a) Give a formula for the number of elements in the 
union of three sets. 

b) Explain why this formula is valid. 

c) Explain how to use the formula from (a) to find the 


number of integers not exceeding 1000 that are divis- 
ible by 6, 10, or 15. 


Supplementary Exercises 


1. 


A group of 10 people begin a chain letter, with each per- 

son sending the letter to four other people. Each of these 

people sends the letter to four additional people. 

a) Find a recurrence relation for the number of letters 
sent at the nth stage of this chain letter, if no person 
ever receives more than one letter. 


b) What are the initial conditions for the recurrence re- 
lation in part (a)? 

c) How many letters are sent at the nth stage of the chain 
letter? 


A nuclear reactor has created 18 grams of a particular ra- 

dioactive isotope. Every hour 1% of this radioactive iso- 

tope decays. 

a) Set up arecurrence relation for the amount of this iso- 
tope left n hours after its creation. 


b) What are the initial conditions for the recurrence re- 
lation in part (a)? 


c) Solve this recurrence relation. 


11. 


12. 


13. 


14. 


15. 


16. 
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d) Explain how to use the formula from (a) to find the 
number of solutions in nonnegative integers to the 
equation x; +X, +X; + x4 = 22 with x, < 8, x < 6, 
and x3; < 5. 

a) Give a formula for the number of elements in the 
union of four sets and explain why it is valid. 

b) Suppose the sets A,, A>, A3, and A, each contain 25 
elements, the intersection of any two of these sets con- 
tains 5 elements, the intersection of any three of these 
sets contains 2 elements, and 1 element is in all four 
of the sets. How many elements are in the union of the 
four sets? 

a) State the principle of inclusion—exclusion. 

b) Outline a proof of this principle. 

Explain how the principle of inclusion—exclusion can be 

used to count the number of onto functions from a set 

with m elements to a set with n elements. 

a) How can you count the number of ways to assign m 
jobs to n employees so that each employee is assigned 
at least one job? 

b) How many ways are there to assign seven jobs to three 
employees so that each employee is assigned at least 
one job? 

Explain how the inclusion—exclusion principle can be 

used to count the number of primes not exceeding the 

positive integer n. 

a) Define a derangement. 

b) Why is counting the number of ways a hatcheck per- 
son can return hats to n people, so that no one receives 
the correct hat, the same as counting the number of 
derangements of n objects? 

c) Explain how to count the number of derangements of 
n objects. 


. Every hour the U.S. government prints 10,000 more $1 


bills, 4000 more $5 bills, 3000 more $10 bills, 2500 more 

$20 bills, 1000 more $50 bills, and the same number of 

$100 bills as it did the previous hour. In the initial hour 

1000 of each bill were produced. 

a) Set up a recurrence relation for the amount of money 
produced in the nth hour. 

b) What are the initial conditions for the recurrence re- 
lation in part (a)? 

c) Solve the recurrence relation for the amount of money 
produced in the nth hour. 

d) Set up a recurrence relation for the total amount of 
money produced in the first n hours. 

e) Solve the recurrence relation for the total amount of 
money produced in the first n hours. 


. Suppose that every hour there are two new bacteria in a 


colony for each bacterium that was present the previous 
hour, and that all bacteria 2 hours old die. The colony 
starts with 100 new bacteria. 
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= 13, 


*14, 


. Solve the recurrence relation a„ = a 


. Find the 


a) Set up a recurrence relation for the number of bacteria 
present after n hours. 

b) What is the solution of this recurrence relation? 

c) When will the colony contain more than | million 
bacteria? 


. Messages are sent over a communications channel using 


two different signals. One signal requires 2 microseconds 

for transmittal, and the other signal requires 3 microsec- 

onds for transmittal. Each signal of a message is followed 

immediately by the next signal. 

a) Find a recurrence relation for the number of different 
signals that can be sent in n microseconds. 

b) What are the initial conditions of the recurrence rela- 
tion in part (a)? 

c) How many different messages can be sent in 12 mi- 
croseconds? 


. A small post office has only 4-cent stamps, 6-cent 


stamps, and 10-cent stamps. Find a recurrence relation 
for the number of ways to form postage of n cents with 
these stamps if the order that the stamps are used mat- 
ters. What are the initial conditions for this recurrence 
relation? 


. How many ways are there to form these postages using 


the rules described in Exercise 6? 


a) 12 cents b) 14 cents 
c) 18 cents d) 22 cents 


. Find the solutions of the simultaneous system of recur- 


rence relations 
ay = Ay} + by 
b, = ân- 7 by 


with dy = 1 and by = 2. 


a o] s 
. Solve the recurrence relation a, =a’, ,/d,_2 if dy = 1 


and a, = 2. [Hint: Take logarithms of both sides to 
obtain a recurrence relation for the sequence log a,, 
n= 0.152; %...5.] 


ay) 

a 
. n n—-1~ n-2 
a, = 2. (See the hint for Exercise 9.) 


if a) = 2 and 


. Find the solution of the recurrence relation a, = 


34,1 — 34,2 + 4,_3 + 1 if aọ = 2,a, = 4, and a, = 8. 
solution of the recurrence relation a, 
= 3a,_, — 3a„-2 + 4,_3 if dg = 2, a; = 2, and a, = 4. 
Suppose that in Example 1 of Section 8.1 a pair of rab- 
bits leaves the island after reproducing twice. Find a re- 
currence relation for the number of rabbits on the island 
in the middle of the nth month. 


In this exercise we construct a dynamic programming al- 
gorithm for solving the problem of finding a subset S 
of items chosen from a set of n items where item 7 has 
a weight w;, which is a positive integer, so that the to- 
tal weight of the items in S is a maximum but does not 
exceed a fixed weight limit W. Let M(j, w) denote the 
maximum total weight of the items in a subset of the 
first j items such that this total weight does not exceed 
w. This problem is known as the knapsack problem. 


a) Show that if w> w, then M(j, w) = M(j — 1, w). 


b) Show that if w; <w, then 
max(M(j — 1, w), w; + M(j — 1, w — w,)). 


M(j,w)= 


c) Use (a) and (b) to construct a dynamic programming 
algorithm for determining the maximum total weight 
of items so that this total weight does not exceed W. 
In your algorithm store the values M( j, w) as they are 
found. 


d) Explain how you can use the values M(j, w) computed 
by the algorithm in part (c) to find a subset of items 
with maximum total weight not exceeding W. 


In Exercises 15-18 we develop a dynamic programming al- 
gorithm for finding a longest common subsequence of two 


SEQUENCES A), Ay, . 


.., a,, and b}, by, ..., b,, an important prob- 


n? 


lem in the comparison of DNA of different organisms. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


Suppose that Ciy Coy ee Cy is 
subsequence of the sequences 
by, by, ..., By. 

a) Show that if a, =b 


a longest common 


a1, az, ...,4, and 


then c, =a, =5, and 


n? p n 
Cp C9, +++, Cp—1 is a longest common subsequence of 
Ay, dy, ... , Am-1 and by, by, ..., b,_, when p > 1. 


b) Suppose that a,, #b,. Show that if c, #a,,, then 

Cys Cay -s Cy is a longest common subsequence of 
Ay, dy, ... , Ay, and b}, bz, ..., b,, and also show that if 

C, $ b, then cj, C2, ...,¢, is a longest common sub- 

sequence of a, dz, ...,,, and b,, bz, ...,b 
Let L(i,j) denote the length of a longest com- 
mon subsequence of aj, a, ...,a; and bj,by,...,b;, 
where 0 <i<m and 0<j <n. Use parts (a) and (b) 
of Exercise 15 to show that L(i,j) satisfies the re- 
currence relation L(i,j)= L(i—1,j—1)+1 if both i 
and j are nonzero and a; = b;, and L(i, j) = max(LGi, j — 
1), L(i — 1, j)) if both i and j are nonzero and a; £ b;, and 
the initial condition L(i, j) = 0 if i = 0 or j = 0. 


n-\" 


Use Exercise 16 to construct a dynamic programming 
algorithm for computing the length of a longest com- 
mon subsequence of two sequences a], d,...,d,, and 
b,, bo, ...,b,, storing the values of L(i, j) as they are 
found. 


n? 


Develop an algorithm for finding a longest com- 
mon subsequence of two sequences a], d),...,d,, and 
b,, bz, ..., b, using the values L(i, j) found by the algo- 
rithm in Exercise 17. 


Find the solution to the recurrence relation f(n) = 
f(n/2) +n? for n = 2* where k is a positive integer and 


fD =1. 


Find the solution to the recurrence relation f(n) = 
3f(n/5) + 2n*, when n is divisible by 5, forn = 5k, where 
k is a positive integer and f(1) = 1. 


Give a big-O estimate for the size of f in Exercise 20 
if f is an increasing function. 


22. 


23. 


24. 


25. 


Find a recurrence relation that describes the number of 
comparisons used by the following algorithm: Find the 
largest and second largest elements of a sequence of n 
numbers recursively by splitting the sequence into two 
subsequences with an equal number of terms, or where 
there is one more term in one subsequence than in the 
other, at each stage. Stop when subsequences with two 
terms are reached. 


Give a big-O estimate for the number of comparisons 
used by the algorithm described in Exercise 22. 


A sequence dj, 4 ,...,d, is unimodal if and only if 
there is an index m, 1 <m <n, such that a; < aj, 
when 1 < i < m and a; > a,,,; when m <i < n. That is, 
the terms of the sequence strictly increase until the mth 
term and they strictly decrease after it, which implies 
that a, is the largest term. In this exercise, a,, will al- 
ways denote the largest term of the unimodal sequence 
Ay, Ay, ..., Ay. 
a) Show that a,, is the unique term of the sequence that 
is greater than both the term immediately preceding it 
and the term immediately following it. 


b) Show that if a,;<a,,; where 1<i<n, 
theni+1<m<n. 
c) Show that if a,>a,, where 1<i<n, 


then 1 <m <i. 

d) Develop a divide-and-conquer algorithm for locat- 
ing the index m. [Hint: Suppose that i<m <j. 
Use parts (a), (b), and (c) to determine whether 
[G4+j)/2] +1<m<n, 1<m< |(i4j)/2] -1, or 
m= (+)/21 

Show that the algorithm from Exercise 24 has worst-case 

time complexity O(log n) in terms of the number of com- 

parisons. 


Let {a,} be a sequence of real numbers. The forward dif- 
ferences of this sequence are defined recursively as fol- 


lows: The first forward difference is Aa, = a,,, — 4 


the 


n? 


(k + Dst forward difference A‘*!a, is obtained from A*a, 


26. 


by AM la, = Afa, — Akay. 
Find Aa,,, where 
a) a,=3. b) a,=4n+7. ©) ap=n°+n+1. 
Let a,, = 3n? +n + 2. Find A‘a,, where k equals 


27. 


* 28. 


29. 


30. 


31. 


a) 2. b) 3. c) 4. 

Suppose that a, = P(n), where P is a polynomial of 
degree d. Prove that A¢+!a, = 0 for all nonnegative inte- 
gers n. 


Let {a,,} and {b,} be sequences of real numbers. Show 
that 


AG, Dn) = apy (Ab,) + b,(Aa,). 


Show that if F(x) and G(x) are the generating func- 
tions for the sequences {a,} and {b,}, respectively, 
and c and d are real numbers, then (cF + dG X(x) is the 
generating function for {ca, + db,}. 


(Requires calculus) This exercise shows how generating 
functions can be used to solve the recurrence relation 


32. 


33. 


34. 


35. 


36. 


37. 


38. 


39. 
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(n+ lay, = 4, + (1/n!) forn > 0 with initial condition 

dy = 1. 

a) Let G(x) be the generating function for {a,}. Show 
that G’(x) = G(x) + e* and G(0) = 1. 

b) Show from part (a) that (e*G())’ = 1, and conclude 
that G(x) = xe* + e*. 

c) Use part (b) to find a closed form for a,,. 

Suppose that 14 students receive an A on the first exam 

in a discrete mathematics class, and 18 receive an A on 

the second exam. If 22 students received an A on either 

the first exam or the second exam, how many students 

received an A on both exams? 


There are 323 farms in Monmouth County that have at 
least one of horses, cows, and sheep. If 224 have horses, 
85 have cows, 57 have sheep, and 18 farms have all three 
types of animals, how many farms have exactly two of 
these three types of animals? 


Queries to a database of student records at a college pro- 
duced the following data: There are 2175 students at the 
college, 1675 of these are not freshmen, 1074 students 
have taken a course in calculus, 444 students have taken 
a course in discrete mathematics, 607 students are not 
freshmen and have taken calculus, 350 students have 
taken calculus and discrete mathematics, 201 students are 
not freshmen and have taken discrete mathematics, and 
143 students are not freshmen and have taken both cal- 
culus and discrete mathematics. Can all the responses to 
the queries be correct? 


Students in the school of mathematics at a university ma- 

jor in one or more of the following four areas: applied 

mathematics (AM), pure mathematics (PM), operations 

research (OR), and computer science (CS). How many 

students are in this school if (including joint majors) there 

are 23 students majoring in AM; 17 in PM; 44 in OR; 63 

in CS; 5 in AM and PM; 8 in AM and CS; 4 in AM and 

OR; 6 in PM and CS; 5 in PM and OR; 14 in OR and CS; 2 

in PM, OR, and CS; 2 in AM, OR, and CS; 1 in PM, AM, 

and OR; 1 in PM, AM, and CS; and 1 in all four fields. 

How many terms are needed when the inclusion- 

exclusion principle is used to express the number of 

elements in the union of seven sets if no more than five 

of these sets have a common element? 

How many solutions in positive integers are there 

to the equation x, +x,+x;=20 with 2<x, <6, 

6 <x, < 10, and O < x, < 5? 

How many positive integers less than 1,000,000 are 

a) divisible by 2, 3, or 5? 

b) not divisible by 7, 11, or 13? 

c) divisible by 3 but not by 7? 

How many positive integers less than 200 are 

a) second or higher powers of integers? 

b) either primes or second or higher powers of integers? 

c) not divisible by the square of an integer greater 
than 1? 

d) not divisible by the cube of an integer greater than 1? 

e) not divisible by three or more primes? 
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* 40. How many ways are there to assign six different jobs to 


41. 


three different employees if the hardest job is assigned 
to the most experienced employee and the easiest job is 
assigned to the least experienced employee? 

What is the probability that exactly one person is given 
back the correct hat by a hatcheck person who gives n 
people their hats back at random? 


Computer Projects 


Write programs with these input and output. 


. Given matrices A}, Aj... 


. Given a positive integer n, list all the moves required in 


the Tower of Hanoi puzzle to move n disks from one peg 
to another according to the rules of the puzzle. 


. Given a positive integer n and an integer k with 1 < k <n, 


list all the moves used by the Frame—Stewart algorithm 
(described in the preamble to Exercise 38 of Section 8.1) 
to move n disks from one peg to another using four pegs 
according to the rules of the puzzle. 


. Given a positive integer n, list all the bit sequences of 


length n that do not have a pair of consecutive Os. 


. Given an integer n greater than 1, write out all ways to 


parenthesize the product of n + 1 variables. 


. Given a set of n talks, their start and end times, and the 


number of attendees at each talk, use dynamic program- 
ming to schedule a subset of these talks in a single lecture 
hall to maximize total attendance. 

.,A,, with dimensions m; X 
My, M, XM3,...,M, X M,4,, respectively, each with in- 
teger entries, use dynamic programming, as outlined 
in Exercise 57 in Section 8.1, to find the mini- 
mum number of multiplications of integers needed to 
compute A,A, + Ap- 


Computations and Explorations 


42. 


43. 


10. 


11. 


12. 


13. 


How many bit strings of length six do not contain four 
consecutive 1s? 

What is the probability that a bit string of length six 
chosen at random contains at least four 1s? 


. Given a recurrence relation a, = C,d,_, + C)4,_7, Where 


cı and c, are real numbers, initial conditions ay = Cy and 
a, = C,, and a positive integer k, find a, using iteration. 


. Given a recurrence relation a, = c)d,_; + C242 and 


initial conditions ay = Cy and a, = C}, determine the 
unique solution. 


. Given a recurrence relation of the form f(n) = af(n/b) + 


c, where a is a real number, b is a positive integer, and c 
is a real number, and a positive integer k, find f(b") using 
iteration. 


Given the number of elements in the intersection of three 
sets, the number of elements in each pairwise intersection 
of these sets, and the number of elements in each set, find 
the number of elements in their union. 


Given a positive integer n, produce the formula for the 
number of elements in the union of n sets. 


Given positive integers m and n, find the number of onto 
functions from a set with m elements to a set with n ele- 
ments. 


Given a positive integer n, list all the derangements of the 
set {1, 2, 3,...,n}. 


Use a computational program or programs you have written to do these exercises. 


. Find the exact value of f199, 509, and fiooo, Where f, is the 


nth Fibonacci number. 


. Find the smallest Fibonacci number greater than 


1,000,000, greater than 1,000,000,000, and greater than 
1,000,000,000,000. 


. Find as many prime Fibonacci numbers as you can. It is 


unknown whether there are infinitely many of these. 


. Write out all the moves required to solve the Tower of 


Hanoi puzzle with 10 disks. 


. Write out all the moves required to use the Frame—Stewart 


algorithm to move 20 disks from one peg to another 
peg using four pegs according to the rules of the Reve’s 
puzzle. 


6. 


Verify the Frame conjecture for solving the Reve’s puzzle 
for n disks for as many integers n as possible by showing 
that the puzzle cannot be solved using fewer moves than 
are made by the Frame—Stewart algorithm with the opti- 
mal choice of k. 


. Compute the number of operations required to multi- 


ply two integers with n bits for various integers n in- 
cluding 16, 64, 256, and 1024 using the fast multiplica- 
tion described in Example 4 of Section 8.3 and the stan- 
dard algorithm for multiplying integers (Algorithm 3 in 
Section 4.2). 


. Compute the number of operations required to multiply 


two n x n matrices for various integers n including 4, 16, 
64, and 128 using the fast matrix multiplication described 


in Example 5 of Section 8.3 and the standard algorithm 
for multiplying matrices (Algorithm 1 in Section 3.3). 

. Find the number of primes not exceeding 10,000 using 
the method described in Section 8.6 to find the number 
of primes not exceeding 100. 


Writing Projects 


Respond to these with essays using outside sources. 


. Find the original source where Fibonacci presented his 
puzzle about modeling rabbit populations. Discuss this 
problem and other problems posed by Fibonacci and give 
some information about Fibonacci himself. 

. Explain how the Fibonacci numbers arise in a variety of 
applications, such as in phyllotaxis, the study of arrange- 
ment of leaves in plants, in the study of reflections by 
mirrors, and so on. 

. Describe different variations of the Tower of Hanoi 
puzzle, including those with more than three pegs (in- 
cluding the Reve’s puzzle discussed in the text and 
exercises), those where disk moves are restricted, and 
those where disks may have the same size. Include what 
is known about the number of moves required to solve 
each variation. 

. Discuss as many different problems as possible where the 
Catalan numbers arise. 

. Discuss some of the problems in which Richard Bellman 
first used dynamic programming. 

. Describe the role dynamic programming algorithms play 
in bioinformatics including for DNA sequence compari- 
son, gene comparison, and RNA structure prediction. 

. Describe the use of dynamic programming in economics 
including its use to study optimal consumption and 
saving. 

. Explain how dynamic programming can be used to solve 
the egg-dropping puzzle which determines which floors 
of a multistory building it is safe to drop eggs from with- 
out breaking. 


10. 
11. 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 
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List all the derangements of {1, 2, 3, 4, 5, 6, 7, 8}. 
Compute the probability that a permutation of n objects 
is a derangement for all positive integers not exceed- 
ing 20 and determine how quickly these probabilities 
approach the number 1 /e. 


. Describe the solution of Ulam’s problem (see Exercise 28 


in Section 8.3) involving searching with one lie found by 
Andrzej Pelc. 


Discuss variations of Ulam’s problem (see Exercise 28 in 
Section 8.3) involving searching with more than one lie 
and what is known about this problem. 


Define the convex hull of a set of points in the plane and 
describe three different algorithms, including a divide- 
and-conquer algorithm, for finding the convex hull of a 
set of points in the plane. 


Describe how sieve methods are used in number theory. 
What kind of results have been established using such 
methods? 


Look up the rules of the old French card game of rencon- 
tres. Describe these rules and describe the work of Pierre 
Raymond de Montmort on le problème de rencontres. 


Describe how exponential generating functions can be 
used to solve a variety of counting problems. 


Describe the Polya theory of counting and the kind of 
counting problems that can be solved using this theory. 


The probléme des ménages (the problem of the house- 
holds) asks for the number of ways to arrange n couples 
around a table so that the sexes alternate and no husband 
and wife are seated together. Explain the method used by 
E. Lucas to solve this problem. 


Explain how rook polynomials can be used to solve count- 
ing problems. 


9.1 Relations and 
Their 
Properties 


9.2 n-ary Relations 
and Their 
Applications 


9.3 Representing 
Relations 


9.4 Closures of 
Relations 


9.5 Equivalence 
Relations 


9.6 Partial 
Orderings 


Relations 


R zion: between elements of sets occur in many contexts. Every day we deal with 
relationships such as those between a business and its telephone number, an employee 
and his or her salary, a person and a relative, and so on. In mathematics we study relationships 
such as those between a positive integer and one that it divides, an integer and one that it is 
congruent to modulo 5, a real number and one that is larger than it, a real number x and the 
value f(x) where f is a function, and so on. Relationships such as that between a program and a 
variable it uses, and that between a computer language and a valid statement in this language, 
often arise in computer science. Relationships between elements of two sets are represented 
using the structure called a binary relation, which is just a subset of the Cartesian product of the 
sets. Relations can be used to solve problems such as determining which pairs of cities are linked 
by airline flights in a network, or finding a viable order for the different phases of a complicated 
project. We will introduce a number of different properties binary relations may enjoy. 

Relationships between elements of more than two sets arise in many contexts. These rela- 
tionships can be represented by n-ary relations, which are collections of n-tuples. Such relations 
are the basis of the relational data model, the most common way to store information in com- 
puter databases. We will introduce the terminology used to study relational databases, define 
some important operations on them, and introduce the database query language SQL. We will 
conclude our brief study of n-ary relations and databases with an important application from 
data mining. In particular, we will show how databases of transactions, represented by n-ary re- 
lations, are used to measure the likelihood that someone buys a particular product from a store 
when they buy one or more other products. 

Two methods for representing relations, using square matrices and using directed graphs, 
consisting of vertices and directed edges, will be introduced and used in later sections of the 
chapter. We will also study relationships that have certain collections of properties that rela- 
tions may enjoy. For example, in some computer languages, only the first 31 characters of the 
name of a variable matter. The relation consisting of ordered pairs of strings in which the first 
string has the same initial 31 characters as the second string is an example of a special type of 
relation, known as an equivalence relation. Equivalence relations arise throughout mathematics 
and computer science. Finally, we will study relations called partial orderings, which general- 
ize the notion of the less than or equal to relation. For example, the set of all pairs of strings of 
English letters in which the second string is the same as the first string or comes after the first 
in dictionary order is a partial ordering. 


Relations and Their Properties 


Links » 


Definition 1 


9.1.1 Introduction 


The most direct way to express a relationship between elements of two sets is to use ordered pairs 
made up of two related elements. For this reason, sets of ordered pairs are called binary relations. 
In this section we introduce the basic terminology used to describe binary relations. Later in 
this chapter we will use relations to solve problems involving communications networks, project 
scheduling, and identifying elements in sets with common properties. 


Let A and B be sets. A binary relation from A to B is a subset of A x B. 
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EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


In other words, a binary relation from A to B is a set R of ordered pairs, where the first element 
of each ordered pair comes from A and the second element comes from B. We use the notation 
a Rb to denote that (a, b) € Randa Ř b to denote that (a, b) ¢ R. Moreover, when (a, b) belongs 
to R, a is said to be related to b by R. 

Binary relations represent relationships between the elements of two sets. We will introduce 
n-ary relations, which express relationships among elements of more than two sets, later in this 
chapter. We will omit the word binary when there is no danger of confusion. 

Examples 1-3 illustrate the notion of a relation. 


Let A be the set of students in your school, and let B be the set of courses. Let R be 
the relation that consists of those pairs (a, b), where a is a student enrolled in course b. 
For instance, if Jason Goodfriend and Deborah Sherman are enrolled in CS518, the pairs 
(Jason Goodfriend, CS518) and (Deborah Sherman, CS518) belong to R. If Jason Goodfriend 
is also enrolled in CS510, then the pair (Jason Goodfriend, CS510) is also in R. However, if 
Deborah Sherman is not enrolled in CS510, then the pair (Deborah Sherman, CS510) is notin R. 

Note that if a student is not currently enrolled in any courses there will be no pairs in R that 
have this student as the first element. Similarly, if a course is not currently being offered there 
will be no pairs in R that have this course as their second element. < 


Let A be the set of cities in the U.S.A., and let B be the set of the 50 states in the U.S.A. 
Define the relation R by specifying that (a,b) belongs to R if a city with name a is in 
the state b. For instance, (Boulder, Colorado), (Bangor, Maine), (Ann Arbor, Michigan), 
(Middletown, New Jersey), (Middletown, New York), (Cupertino, California), and 
(Red Bank, New Jersey) are in R. < 


Let A = {0, 1, 2} and B = {a, b}. Then {(0, a), (0, b), (1, a), (2, b)} is a relation from A to B. This 
means, for instance, that 0 Ra, but that 1 # b. Relations can be represented graphically, as shown 
in Figure 1, using arrows to represent ordered pairs. Another way to represent this relation is to 
use a table, which is also done in Figure 1. We will discuss representations of relations in more 


detail in Section 9.3. < 
0e 
e R 4 b 
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FIGURE 1 Displaying the ordered pairs in the relation R from Example 3. 


9.1.2 Functions as Relations 


Recall that a function f from a set A to a set B (as defined in Section 2.3) assigns exactly 
one element of B to each element of A. The graph of f is the set of ordered pairs (a, b) such 
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that b = f(a). Because the graph of f is a subset of A x B, it is a relation from A to B. Moreover, 
the graph of a function has the property that every element of A is the first element of exactly 
one ordered pair of the graph. 

Conversely, if R is a relation from A to B such that every element in A is the first element 
of exactly one ordered pair of R, then a function can be defined with R as its graph. This can 
be done by assigning to an element a of A the unique element b € B such that (a, b) E€ R. (Note 
that the relation R in Example 2 is not the graph of a function because Middletown occurs more 
than once as the first element of an ordered pair in R.) 

A relation can be used to express a one-to-many relationship between the elements of the 
sets A and B (as in Example 2), where an element of A may be related to more than one element 
of B. A function represents a relation where exactly one element of B is related to each element 
of A. 

Relations are a generalization of graphs of functions; they can be used to express a much 
wider class of relationships between sets. (Recall that the graph of the function f from A to B is 
the set of ordered pairs (a, f(a)) for a € A.) 


9.1.3 Relations on a Set 


Relations from a set A to itself are of special interest. 


A relation on a set A is a relation from A to A. 


In other words, a relation on a set A is a subset of A x A. 


Let A be the set {1, 2, 3,4}. Which ordered pairs are in the relation R = {(a, b) | a divides b}? 


Solution: Because (a, b) is in R if and only if a and b are positive integers not exceeding 4 such 
that a divides b, we see that 


R= {(1, 1), (1, 2), (1, 3), (1, 4), (2, 2), (2, 4), (3, 3), (4, 4}- 


The pairs in this relation are displayed both graphically and in tabular form in Figure 2. < 
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FIGURE 2 Displaying the ordered pairs in 
the relation R from Example 4. 
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Next, some examples of relations on the set of integers will be given in Example 5. 
Consider these relations on the set of integers: 


R, = {(a, b) | a < b}, 

R, = {(a,b) | a> b}, 

R, = {(a, b) | a = b ora = =b}, 
R, = {(a,b) |a =b}, 

R; = {(a,b)|a=b+1}, 

R = {(a,b)|a+b <3}. 


Which of these relations contain each of the pairs (1, 1), (1, 2), (2, 1), (1, —1), and (2, 2)? 


Remark: Unlike the relations in Examples 1—4, these are relations on an infinite set. 


Solution: The pair (1, 1) is in R}, R3, Ry, and Re; (1, 2) is in R, and Rg; (2, 1) is in R}, Rs, and 
Re; (1, —1) is in R}, R3, and Rg; and finally, (2, 2) is in R4, R3, and R4. < 


It is not hard to determine the number of relations on a finite set, because a relation on a 
set A is simply a subset of A x A. 


How many relations are there on a set with n elements? 


Solution: A relation on a set A is a subset of A x A. Because A x A has n? elements when A has 
n elements, and a set with m elements has 2” subsets, there are 2” subsets of A X A. Thus, there 
are 2” relations on a set with n elements. For example, there are 2? = 2 = 512 relations on 
the set {a, b, c}. < 


9.1.4 Properties of Relations 


There are several properties that are used to classify relations on a set. We will introduce the 
most important of these here. (You may find it instructive to study this material with the contents 
of Section 9.3. In that section, several methods for representing relations will be introduced that 
can help you understand each of the properties that we introduce here.) 

In some relations an element is always related to itself. For instance, let R be the relation on 
the set of all people consisting of pairs (x, y) where x and y have the same mother and the same 
father. Then xRx for every person x. 


A relation R on a set A is called reflexive if (a, a) E R for every element a E A. 


Remark: Using quantifiers we see that the relation R on the set A is reflexive if Va((a, a) E R), 
where the universe of discourse is the set of all elements in A. 


We see that a relation on A is reflexive if every element of A is related to itself. 
Examples 7-9 illustrate the concept of a reflexive relation. 


EXAMPLE 7 


EXAMPLE 8 
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Consider the following relations on {1, 2, 3, 4}: 


R, = {(1, 1), (1, 2), (2, 1), (2, 2), 3, 4}, (4, D, 4 49}, 

R, = {(1, 1), (1, 2), (2, D}, 

R, = {(1, 1), (1, 2), (1, 4), (2, 1), (2, 2), 3, 3), (4, 1), (4, 4)}, 

R, = {(2, 1), 3, 1), (3, 2), (4, 1), (4, 2), (4, 3)}, 

R; = {C, 1), (1, 2), (, 3), d, 4, (2, 2), (2, 3), (2, 4), 63, 3), G, 4), (4, 4)}, 
Rs = {(3, 4)}. 


Which of these relations are reflexive? 


Solution: The relations R} and R; are reflexive because they both contain all pairs of the form 
(a, a), namely, (1, 1), (2, 2), (3, 3), and (4, 4). The other relations are not reflexive because they do 
not contain all of these ordered pairs. In particular, R,, Rj, R4, and Re are not reflexive because 
(3, 3) is not in any of these relations. < 


Which of the relations from Example 5 are reflexive? 


Solution: The reflexive relations from Example 5 are R, (because a < a for every integer a), 
R3, and R,. For each of the other relations in this example it is easy to find a pair of the 
form (a, a) that is not in the relation. (This is left as an exercise for the reader.) < 


Is the “divides” relation on the set of positive integers reflexive? 


Solution: Because a | a whenever a is a positive integer, the “divides” relation is reflexive. (Note 
that if we replace the set of positive integers with the set of all integers the relation is not reflexive 
because by definition 0 does not divide 0.) < 


In some relations an element is related to a second element if and only if the second element 
is also related to the first element. The relation consisting of pairs (x, y), where x and y are 
students at your school with at least one common class has this property. Other relations have 
the property that if an element is related to a second element, then this second element is not 
related to the first. The relation consisting of the pairs (x, y), where x and y are students at your 
school, where x has a higher grade point average than y has this property. 


A relation R on a set A is called symmetric if (b, a) E R whenever (a, b) E€ R, for alla, b € A. 
A relation R on a set A such that for all a, b E A, if (a, b) E R and (b, a) E R, then a = b is 
called antisymmetric. 


Remark: Using quantifiers, we see that the relation R on the set A is symmetric if 
VaVb((a, b) E R > (b,a) € R). Similarly, the relation R on the set A is antisymmetric if 
VaVb(((a, b) E RA (b, a) E€ R) > (a = b)). 


In other words, a relation is symmetric if and only if a is related to b always implies that 
b is related to a. For instance, the equality relation is symmetric because a = b if and only if 
b = a. A relation is antisymmetric if and only if there are no pairs of distinct elements a and b 
with a related to b and b related to a. That is, the only way to have a related to b and b related 
to a is for a and b to be the same element. For instance, the less than or equal to relation is 
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antisymmetric. To see this, note that a < b and b < a implies that a = b. The terms symmetric 
and antisymmetric are not opposites, because a relation can have both of these properties or may 
lack both of them (see Exercise 10). A relation cannot be both symmetric and antisymmetric if 
it contains some pair of the form (a, b) in which a ¥ b. 


Remark: Although relatively few of the 2” relations on a set with n elements are symmetric 
or antisymmetric, as counting arguments can show, many important relations have one of these 
properties. (See Exercise 49.) 


Which of the relations from Example 7 are symmetric and which are antisymmetric? 


Solution: The relations R, and R, are symmetric, because in each case (b, a) belongs to the re- 
lation whenever (a, b) does. For R,, the only thing to check is that both (2, 1) and (1, 2) are in 
the relation. For R}, it is necessary to check that both (1, 2) and (2, 1) belong to the relation, 
and (1, 4) and (4, 1) belong to the relation. The reader should verify that none of the other 
relations is symmetric. This is done by finding a pair (a, b) such that it is in the relation 
but (b, a) is not. 

R,, Rs, and Re are all antisymmetric. For each of these relations there is no pair of elements 
a and b with a £ b such that both (a, b) and (b, a) belong to the relation. The reader should 
verify that none of the other relations is antisymmetric. This is done by finding a pair (a, b) with 
a # b such that (a, b) and (b, a) are both in the relation. 


Which of the relations from Example 5 are symmetric and which are antisymmetric? 


Solution: The relations R}, R4, and Rẹ are symmetric. R} is symmetric, for if a = b or a= —b, 
then b = a or b = —a. R, is symmetric because a = b implies that b = a. Rẹ is symmetric be- 
cause a+ b < 3 implies that b + a < 3. The reader should verify that none of the other relations 
is symmetric. 

The relations R,, Rj, R4, and R; are antisymmetric. R, is antisymmetric because the in- 
equalities a < b and b <a imply that a= b. R, is antisymmetric because it is impossible 
that a > b and b > a. R, is antisymmetric, because two elements are related with respect to 
R, if and only if they are equal. R; is antisymmetric because it is impossible that a = b + 1 
and b = a + 1. The reader should verify that none of the other relations is antisymmetric. <4 


Is the “divides” relation on the set of positive integers symmetric? Is it antisymmetric? 


Solution: This relation is not symmetric because 1 |2, but 2 1. However, it is antisymmetric. To 
see this, note that if a and b are positive integers with a| b and b |a, then a = b (the verification 
of this is left as an exercise for the reader). < 


Let R be the relation consisting of all pairs (x, y) of students at your school, where x has 
taken more credits than y. Suppose that x is related to y and y is related to z. This means 
that x has taken more credits than y and y has taken more credits than z. We can conclude 
that x has taken more credits than z, so that x is related to z. What we have shown is that R has 
the transitive property, which is defined as follows. 


A relation R on a set A is called transitive if whenever (a,b) ER and (b,c) ER, 
then (a, c) E€ R, for all a, b, c E€ A. 


EXAMPLE 13 


Extra » 
Examples 


EXAMPLE 14 


EXAMPLE 15 


EXAMPLE 16 


9.1 Relations and Their Properties 605 


Remark: Using quantifiers we see that the relation R on a set A is transitive if we have 
VaVbVc(((a, b) E RA (b, c) E R) > (a,c) E R). 


Which of the relations in Example 7 are transitive? 


Solution: Ry, Rs, and Rg are transitive. For each of these relations, we can show that it is transitive 
by verifying that if (a, b) and (b, c) belong to this relation, then (a, c) also does. For instance, R4 
is transitive, because (3, 2) and (2, 1), (4, 2) and (2, 1), (4, 3) and (3, 1), and (4, 3) and (3, 2) are 
the only such sets of pairs, and (3, 1), (4, 1), and (4, 2) belong to R,. The reader should verify 
that R; and R, are transitive. 

R, is not transitive because (3,4) and (4, 1) belong to R,, but (3,1) does not. R, is 
not transitive because (2, 1) and (1,2) belong to R,, but (2, 2) does not. R} is not transitive 
because (4, 1) and (1, 2) belong to R}, but (4, 2) does not. < 


Which of the relations in Example 5 are transitive? 


Solution: The relations R,, Ry, R3, and R, are transitive. R} is transitive because a < b and b < c 
imply that a < c. R, is transitive because a > b and b > c imply that a > c. R} is transitive 
because a = +b and b = +c imply that a = +c. R, is clearly transitive, as the reader should 
verify. R; is not transitive because (2, 1) and (1, 0) belong to R,, but (2, 0) does not. Rg is not 
transitive because (2, 1) and (1, 2) belong to Rg, but (2, 2) does not. < 


Is the “divides” relation on the set of positive integers transitive? 


Solution: Suppose that a divides b and b divides c. Then there are positive integers k and / 
such that b = ak and c = bl. Hence, c = a(kl), so a divides c. It follows that this relation is 
transitive. < 


We can use counting techniques to determine the number of relations with specific proper- 
ties. Finding the number of relations with a particular property provides information about how 
common this property is in the set of all relations on a set with n elements. 


How many reflexive relations are there on a set with n elements? 


Solution: A relation R on a set A is a subset of A X A. Consequently, a relation is determined by 
specifying whether each of the n? ordered pairs in A x A is in R. However, if R is reflexive, each 
of the n ordered pairs (a, a) for a € A must be in R. Each of the other n(n — 1) ordered pairs of 
the form (a, b), where a + b, may or may not be in R. Hence, by the product rule for counting, 
there are 2”("-) reflexive relations [this is the number of ways to choose whether each element 
(a, b), with a £ b, belongs to R]. < 


Formulas for the number of symmetric relations and the number of antisymmetric rela- 
tions on a set with n elements can be found using reasoning similar to that in Example 16 
(see Exercise 49). However, no general formula is known that counts the transitive relations on a 
set with n elements. Currently, T(n), the number of transitive relations on a set with n elements, is 
known only for 0 < n < 18. For example, T(4) = 3,994, T(5) = 154,303, and T(6) = 9,415,189. 
(The values of T(n) for n = 0, 1, 2, ..., 18, are the terms of the sequence A006905 in the OEIS, 
which is discussed in Section 2.4.) 
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9.1.5 Combining Relations 


Because relations from A to B are subsets of A x B, two relations from A to B can be combined 
in any way two sets can be combined. Consider Examples 17-19. 


Let A= {1,2,3} and B= {1,2,3,4}. The relations R, = {(1,1), (2,2), (3,3)} and 
R, = {(1, 1), (1, 2), (1, 3), (1, 4)} can be combined to obtain 


R, UR, = {(1, 1), (L, 2), CL, 3), CL 4), @, 2), G, 3)}, 

R, NR, = {(L 1)}, 

R; — R, = {(2, 2), GB, 3)}, 

Ry = Ry = {(1, 2, (1, 3), d, 4)}. < 


Let A and B be the set of all students and the set of all courses at a school, respectively. Sup- 
pose that R, consists of all ordered pairs (a, b), where a is a student who has taken course b, 
and R, consists of all ordered pairs (a, b), where a is a student who requires course b to graduate. 
What are the relations R; U R3, Ri A R3, Ri ® Ry, R; — Ro, and R, — R,? 


Solution: The relation R, U R, consists of all ordered pairs (a, b), where a is a student who either 
has taken course b or needs course b to graduate, and R, N R, is the set of all ordered pairs (a, b), 
where a is a student who has taken course b and needs this course to graduate. Also, R) ® R, 
consists of all ordered pairs (a, b), where student a has taken course b but does not need it to 
graduate or needs course b to graduate but has not taken it. R} — R, is the set of ordered pairs 
(a, b), where a has taken course b but does not need it to graduate; that is, b is an elective course 
that a has taken. R, — R, is the set of all ordered pairs (a, b), where b is a course that a needs to 
graduate but has not taken. < 


Let R, be the less than relation on the set of real numbers and let R, be the greater than relation on 
the set of real numbers, that is, R; = {(x, y) | x < y} and R, = {(x, y) | x > y}. What are R, UR), 
R, A R3, R; — Ry, R, — R}, and R; ® Ry? 


Solution: We note that (x, y) E€ R, U R, if and only if (x, y) € R, or (x, y) € R,. Hence, (x, y) € 
R, UR, if and only if x < y or x > y. Because the condition x < y or x > y is the same as the 
condition x # y, it follows that R} U R, = {(x, y) | x # y}. In other words, the union of the less 
than relation and the greater than relation is the not equals relation. 

Next, note that it is impossible for a pair (x, y) to belong to both R, and R, because it is 
impossible that x < y and x > y. It follows that R} N R, = Ø. We also see that R} — R, = Rj, 
R, —R, = R}, and Ri ®R, =R, UR, -R NR, = {@, y) |x Fy}. < 


There is another way that relations are combined that is analogous to the composition of 
functions. 


Let R be a relation from a set A to a set B and S a relation from B to a set C. The composite 
of R and S is the relation consisting of ordered pairs (a, c), where a € A, c € C, and for which 
there exists an element b € B such that (a, b) € R and (b, c) € S. We denote the composite 
of R and S by SOR. 


EXAMPLE 20 


EXAMPLE 21 


Definition 7 


EXAMPLE 22 


9.1 Relations and Their Properties 607 


Computing the composite of two relations requires that we find elements that are the second 
element of ordered pairs in the first relation and the first element of ordered pairs in the second 
relation, as Examples 20 and 21 illustrate. 


What is the composite of the relations R and S, where R is the relation from {1, 2,3} to 
{1, 2, 3,4} with R = {(1, 1), (1, 4), (2, 3), (3, 1), (3, 4)} and S is the relation from {1, 2, 3, 4} to 
{0, 1,2} with S = {(1, 0), (2, 0), (3, 1), (3, 2), (4, 1)}? 


Solution: SOR is constructed using all ordered pairs in R and ordered pairs in S, where the 
second element of the ordered pair in R agrees with the first element of the ordered pair 
in S. For example, the ordered pairs (2, 3) in R and (3, 1) in S produce the ordered pair (2, 1) 
in S °R. Computing all the ordered pairs in the composite, we find 


SOR= {(1, 0), C, 1), (2, 1, (2, 2), (3, 0), (3, D}- 


Figure 3 illustrates how this composition is found. In the figure, we examine all paths that travel 
via two directed edges from the leftmost elements to the rightmost elements via an element in 
the middle. 


R S 
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1 0 151-0 (1,0 
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5 2 i 23331 (2,1) 

3 233-52 (2,2) 

33130 (3,0) 

à 2 33451 (3,1) 
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FIGURE3 Constructing S ° R. 


Composing the Parent Relation with Itself Let R be the relation on the set of all people 
such that (a, b) € R if person a is a parent of person b. Then (a, c) € ROR if and only if there 
is a person b such that (a, b) € R and (b, c) € R, that is, if and only if there is a person b such 
that a is a parent of b and b is a parent of c. In other words, (a, c) € ROR if and only if a is a 
grandparent of c. < 


The powers of a relation R can be recursively defined from the definition of a composite of 
two relations. 


Let R be a relation on the set A. The powers R”, n = 1, 2,3, ..., are defined recursively by 


R'=R and = R™*1 = R" oR, 


The definition shows that R? = R © R, R? = R? o R = (ROR)OR, and so on. 
Let R = {(1, 1), (2, 1), (3, 2), (4, 3)}. Find the powers R”, n = 2,3,4,.... 


Solution: Because R? = ROR, we find that R? = {(1, 1), (2, 1), (3, 1), (4, 2)}. Furthermore, be- 
cause R? = R? oR, R? = {(1, 1), (2, 1), (3, 1), 4 1)}. Additional computation shows that R* 
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is the same as R?, so R* = {(1, 1), (2, 1), (3, 1), (4 1)}. It also follows that R” = R? for n = 
5, 6, 7, .... The reader should verify this. < 


The following theorem shows that the powers of a transitive relation are subsets of this 
relation. It will be used in Section 9.4. 


The relation R on a set A is transitive if and only if R” C R for n = 1, 2,3,.... 


Proof: We first prove the “if’ part of the theorem. We suppose that R” C R for n= 1, 
2,3, .... In particular, R? CR. To see that this implies R is transitive, note that if (a,b) ER 
and (b, c) E R, then by the definition of composition, (a, c) € R?. Because R? C R, this means 
that (a, c) E R. Hence, R is transitive. 

We will use mathematical induction to prove the only if part of the theorem. Note that this 
part of the theorem is trivially true for n = 1. 

Assume that R” C R, where n is a positive integer. This is the inductive hypothesis. To 
complete the inductive step we must show that this implies that R”+! is also a subset of R. 
To show this, assume that (a, b) € R”t!. Then, because R”+! = R” o R, there is an element x 
with x € A such that (a, x) E R and (x, b) € R”. The inductive hypothesis, namely, that R” C R, 
implies that (x, b) € R. Furthermore, because R is transitive, and (a, x) E R and (x, b) E€ R, it 
follows that (a, b) € R. This shows that R’*! C R, completing the proof. < 


1. List the ordered pairs in the relation R from 5. Determine whether the relation R on the set of all 


A = {0, 1,2,3,4} to B = {0, 1, 2,3}, where (a, b) E€ R 

if and only if 

a) a=b. 

c) a>b. d) a |b. 

e) gcd(a, b) = 1. f) Icm(a, b) = 2. 

. a) List all the ordered pairs in the relation 
R = {(a, b) | a divides b} on the set {1, 2, 3, 4, 5, 6}. 

b) Display this relation graphically, as was done in 
Example 4. 

c) Display this relation in tabular form, as was done in 
Example 4. 


b) a+b=4. 


. For each of these relations on the set {1, 2, 3, 4}, decide 
whether it is reflexive, whether it is symmetric, whether 
it is antisymmetric, and whether it is transitive. 

a) {(2, 2), 2, 3), 2, 4) G, 2) G, 3), 3,4} 

b) {CL 1), (1, 2), 2 1), @ 2), G, 3), (4 4} 

c) {(2,4), (4 2)} 

d) {(1, 2), (2 3), 3,4} 

e) {( 1), @ 2), G, 3), (4, 4)} 

f) {0 3). 4), 2 3),(24,3, D, 3,9} 

. Determine whether the relation R on the set of all people 
is reflexive, symmetric, antisymmetric, and/or transitive, 
where (a, b) € R if and only if 

a) ais taller than b. 

b) a and b were born on the same day. 

c) ahas the same first name as b. 

d) a and b have a common grandparent. 


Web pages is reflexive, symmetric, antisymmetric, and/or 
transitive, where (a, b) € R if and only if 


a) everyone who has visited Web page a has also visited 
Web page b. 

b) there are no common links found on both Web 
page a and Web page b. 

c) there is at least one common link on Web page a and 
Web page b. 

d) there is a Web page that includes links to both Web 
page a and Web page b. 


. Determine whether the relation R on the set of all real 


numbers is reflexive, symmetric, antisymmetric, and/or 
transitive, where (x, y) € R if and only if 


a) x+y=0. b) x = +y. 
c) x— yis a rational number. 

d) x= 2y. e) x 20. 
f) xy =0. g) x=l1. 


h) x=lory=1. 


. Determine whether the relation R on the set of all integers 


is reflexive, symmetric, antisymmetric, and/or transitive, 
where (x, y) € R if and only if 

a) x#y. b) xy > 1. 

c) x=ytlorx=y-1. 

d) x = y (mod 7). e) xis a multiple of y. 

f) x and y are both negative or both nonnegative. 

g) x=y’. h) x> y. 


. Show that the relation R = Ø on a nonempty set S is sym- 


metric and transitive, but not reflexive. 


. Show that the relation R = Ø on the empty set S = Ø is 


reflexive, symmetric, and transitive. 


10. 


Give an example of a relation on a set that is 


a) both symmetric and antisymmetric. 
b) neither symmetric nor antisymmetric. 


A relation R on the set A is irreflexive if for every 
a E A, (aa) € R. That is, R is irreflexive if no element 
in A is related to itself. 


11. 
12. 
13. 
14. 
15. 
16. 


17. 


Which relations in Exercise 3 are irreflexive? 

Which relations in Exercise 4 are irreflexive? 

Which relations in Exercise 5 are irreflexive? 

Which relations in Exercise 6 are irreflexive? 

Can arelation on a set be neither reflexive nor irreflexive? 
Use quantifiers to express what it means for a relation to 
be irreflexive. 

Give an example of an irreflexive relation on the set of 
all people. 


A relation R is called asymmetric if (a, b) € R implies that 
(b,a) £ R. Exercises 18-24 explore the notion of an asym- 
metric relation. Exercise 22 focuses on the difference between 
asymmetry and antisymmetry. 


18. 
19. 
20. 
21. 
22. 


23. 


24. 


25. 


Which relations in Exercise 3 are asymmetric? 
Which relations in Exercise 4 are asymmetric? 
Which relations in Exercise 5 are asymmetric? 
Which relations in Exercise 6 are asymmetric? 


Must an asymmetric relation also be antisymmetric? 
Must an antisymmetric relation be asymmetric? Give rea- 
sons for your answers. 


Use quantifiers to express what it means for a relation to 
be asymmetric. 

Give an example of an asymmetric relation on the set of 
all people. 

How many different relations are there from a set with m 
elements to a set with n elements? 


Cs Let R be a relation from a set A to a set B. The inverse rela- 
tion from B to A, denoted by R7!, is the set of ordered pairs 
{(b, a) | (a,b) E R}. The complementary relation R is the 
set of ordered pairs {(a, b) | (a, b) € R}. 


26. 


27. 


28. 


29. 


30. 


Let R be the relation R = {(a, b) | a < b} on the set of 
integers. Find 

a) R. b) R. 

Let R be the relation R = {(a, b) | a divides b} on the set 
of positive integers. Find 

a) R`. b) R. 

Let R be the relation on the set of all states in the United 
States consisting of pairs (a, b) where state a borders state 
b. Find 

a) RT. b) R. 

Suppose that the function f from A to B is a one-to- 
one correspondence. Let R be the relation that equals the 
graph of f. That is, R = {(a, f(a)) | a € A}. What is the 
inverse relation R7!? 

Let R, = {(1, 2), (2, 3), (3, 4)} and R, = {(1, 1), (1, 2), 
(2, 1), (2, 2), (2, 3), (3, 1), (3, 2), (3, 3), (3, 4) } be relations 
from {1, 2, 3} to {1, 2, 3, 4}. Find 
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a) R UR. b) RAR. 
ce) R —R. d) R- R}. 

31. Let A be the set of students at your school and B the set 
of books in the school library. Let R, and R, be the rela- 
tions consisting of all ordered pairs (a, b), where student 
a is required to read book b in a course, and where stu- 
dent a has read book b, respectively. Describe the ordered 
pairs in each of these relations. 


a) R UR, b) RINR, 
c) ROR d) Ri- R, 
e) R -R 


32. Let R be the relation {(1, 2), (1, 3), (2, 3), (2, 4), (3, D}, 
and let S be the relation {(2, 1), (3, 1), (3, 2), (4, 2)}. Find 
SOR. 

33. Let R be the relation on the set of people consisting of 
pairs (a, b), where a is a parent of b. Let S be the rela- 
tion on the set of people consisting of pairs (a, b), where 
a and b are siblings (brothers or sisters). What are S ° R 
and R ° $? 

Exercises 34—38 deal with these relations on the set of real 

numbers: 


R, = {(a,b) € R? | a> b}, the greater than relation, 

R, = {(a, b) € R? | a > b}, the greater than or equal to 
relation, 

R, = {(a, b) € R? | a < b}, the less than relation, 


R, = {(a, b) € R? | a < b}, the less than or equal to 
relation, 


R; = {(a, b) E€ R? | a = b}, the equal to relation, 
Ro = {(a, b) € R? | a + b}, the unequal to relation. 


34. Find 
a) R, U R3. b) R, URs. 
c) Ry AR}. d) R3 N Rs. 
e) Ri — R. f) R -R}. 
g) Ri ® R3. h) R, ® Ry. 
35. Find 
a) R, U R4. b) R; U Rẹ. 
c) R; N Re- d) R} N Re- 
e) Ry — Re. f) R- R. 
g) R OR. h) R; @Rs. 
36. Find 
a) R oR). b) R, oR). 
©) R, oR. d) R; oR}. 
e) R, ORs. f) R, oR. 
g) Ro R3. h) R; 0 R3. 
37. Find 
a) RoR. b) R, oR). 
c) R} o R;. d) R,oR,. 
e) R; 0R}. f) R30 Rẹ 


g) R,oRg. h) R; 0 Rg. 
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38. 
39. 


Find the relations R? for i = 1, 2,3, 456 
Find the relations s? for i = 1, 2, 3, 4, 5, 6 where 


S, = {(a, b) € Z? | a > b}, the greater than relation, 
S, = {(a, b) € Z? | a > b}, the greater than or equal to 


relation, 


S3 = {(a, b) € Z? | a < b}, the less than relation, 
S, = {(a, b) € Z? | a < b}, the less than or equal to 


relation, 


Ss = {(a, b) € Z? | a = b}, the equal to relation, 
S6 = {(a, b) E€ Z | a + b}, the unequal to relation. 


40. 


41. 


42. 


43. 


44. 
45. 


46. 


47. 


48. 


Let R be the parent relation on the set of all people (see 
Example 21). When is an ordered pair in the relation R?? 


Let R be the relation on the set of people with doctorates 
such that (a, b) € R if and only if a was the thesis advi- 
sor of b. When is an ordered pair (a, b) in R*? When is 
an ordered pair (a, b) in R”, when n is a positive integer? 
(Assume that every person with a doctorate has a thesis 
advisor.) 

Let R; and R, be the “divides” and “is a multiple of” 
relations on the set of all positive integers, respectively. 
That is, R, = {(a, b) | a divides b} and R, = {(a, b) | a 
is a multiple of b}. Find 


a) Ri UR). b) Ri NR. 
c) R, — R. d) R —R,. 
e) Ri @ R. 


Let R, and R, be the “congruent modulo 3” and the “con- 
gruent modulo 4” relations, respectively, on the set of in- 
tegers. That is, R; = {(a, b) | a =b(mod 3)} and R, = 
{(a, b) | a = b (mod 4)}. Find 


a) Ri UR. b) RAR. 
c) R, — R. d) R, —R,. 
e) Ri ®R,. 


List the 16 different relations on the set {0, 1}. 

How many of the 16 different relations on {0, 1} contain 

the pair (0, 1)? 

Which of the 16 relations on {0, 1}, which you listed in 

Exercise 44, are 

a) reflexive? b) irreflexive? 

c) symmetric? d) antisymmetric? 

e) asymmetric? f) transitive? 

a) How many relations are there on the set {a, b, c, d}? 

b) How many relations are there on the set {a, b, c, d} 
that contain the pair (a, a)? 


Let S be a set with n elements and let a and b be dis- 
tinct elements of S. How many relations R are there on S 
such that 


a) (a,b) E€ R? b) (a, b) R? 
c) no ordered pair in R has a as its first element? 
d) at least one ordered pair in R has a as its first element? 


e) no ordered pair in R has a as its first element or b as 
its second element? 


* 49, 


* 50. 


51. 


52. 


53. 


54. 


55. 


56. 


57. 


58. 


59. 


*60. 


61. 


62. 


f) at least one ordered pair in R either has a as its first 
element or has b as its second element? 

How many relations are there on a set with n elements 

that are 

a) symmetric? 

c) asymmetric? 

e) reflexive and symmetric? 

f) neither reflexive nor irreflexive? 


b) antisymmetric? 
d) irreflexive? 


How many transitive relations are there on a set with n 
elements if 
a) n=1? 


b) n= 2? c) n=3? 


Find the error in the “proof” of the following “theorem.” 


“Theorem”: Let R be a relation on a set A that is symmet- 
ric and transitive. Then R is reflexive. 


“Proof”: Let a E€ A. Take an element b € A such that 
(a, b) E R. Because R is symmetric, we also have (b, a) € 
R. Now using the transitive property, we can conclude 
that (a, a) € R because (a, b) € R and (b, a) € R. 


Suppose that R and S are reflexive relations on a set A. 
Prove or disprove each of these statements. 

a) RUS is reflexive. 

b) ROS is reflexive. 

c) R@ Sis irreflexive. 

d) R- S is irreflexive. 

e) SOR is reflexive. 


Show that the relation R on a set A is symmetric if and 
only if R = R7', where R7! is the inverse relation. 


Show that the relation R on a set A is antisymmetric if 
and only if R N R7! is a subset of the diagonal relation 
A= {(a a) |a E A}. 


Show that the relation R on a set A is reflexive if and only 
if the inverse relation R7! is reflexive. 


Show that the relation R on a set A is reflexive if and only 
if the complementary relation R is irreflexive. 


Let R be a relation that is reflexive and transitive. Prove 
that R” = R for all positive integers n. 


Let R be the relation on the set {1, 2, 3, 4, 5} containing 
the ordered pairs (1, 1), (1, 2), (1, 3), (2, 3), (2, 4), (3, 1), 
(3, 4), (3, 5), (4, 2), (4, 5), (5, 1), (5, 2), and (5, 4). Find 

a) R. b) RÈ. c) Rt. d) R5. 


Let R be a reflexive relation on a set A. Show that R” is 
reflexive for all positive integers n. 

Let R be a symmetric relation. Show that R” is symmetric 
for all positive integers n. 


Suppose that the relation R is irreflexive. Is R? necessar- 
ily irreflexive? Give a reason for your answer. 


Derive a big-O estimate for the number of integer com- 
parisons needed to count all transitive relations on a set 
with n elements using the brute force approach of check- 
ing every relation of this set for transitivity. 
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9.2 n-ary Relations and Their Applications 


Definition 1 


EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


9.2.1 Introduction 


Relationships among elements of more than two sets often arise. For instance, there is a relation- 
ship involving the name of a student, the student’s major, and the student’s grade point average. 
Similarly, there is a relationship involving the airline, flight number, starting point, destination, 
departure time, and arrival time of a flight. An example of such a relationship in mathematics 
involves three integers, where the first integer is larger than the second integer, which is larger 
than the third. Another example is the betweenness relationship involving points on a line, such 
that three points are related when the second point is between the first and the third. 

We will study relationships among elements from more than two sets in this section. 
These relationships are called n-ary relations. These relations are used to represent com- 
puter databases. These representations help us answer queries about the information stored in 
databases, such as: Which flights land at O’ Hare Airport between 3 A.M. and 4 A.M.? Which 
students at your school are sophomores majoring in mathematics or computer science and have 
greater than a 3.0 average? Which employees of a company have worked for the company less 
than 5 years and make more than $50,000? 


9.2.2 n-ary Relations 


We begin with the basic definition on which the theory of relational databases rests. 


Let A,,A,,...,A,, be sets. An n-ary relation on these sets is a subset of A, X A, X + XA,. 
The sets A}, Aj, ..., A,, are called the domains of the relation, and n is called its degree. 


Let R be the relation on N x N x N consisting of triples (a, b, c), where a, b, and c are integers 
witha < b < c. Then (1, 2, 3) € R, but (2, 4, 3) £ R. The degree of this relation is 3. Its domains 
are all equal to the set of natural numbers. < 


Let R be the relation on Zx Z x Z consisting of all triples of integers (a, b,c) in which 
a, b, and c form an arithmetic progression. That is, (a,b,c) € R if and only if there is 
an integer k such that b = a+ k and c =a + 2k, or equivalently, such that b — a = k and 
c — b = k. Note that (1, 3, 5) E R because 3 = 1 + 2 and 5 = 1 + 2 - 2, but (2, 5, 9) € R because 
5 — 2 = 3 while 9 — 5 = 4. This relation has degree 3 and its domains are all equal to the set of 
integers. < 


Let R be the relation on Z x Z x Z* consisting of triples (a, b, m), where a, b, and m are inte- 
gers with m > 1 and a = b (mod m). Then (8, 2, 3), (—1, 9, 5), and (14, 0, 7) all belong to R, but 
(7, 2, 3), (—2, —8, 5), and (11, 0, 6) do not belong to R because 8 = 2 (mod 3), —1 = 9 (mod 5), 
and 14 = 0 (mod 7), but 7 Æ 2 (mod 3), —2 Æ —8 (mod 5), and 11 # 0 (mod 6). This relation has 
degree 3 and its first two domains are the set of all integers and its third domain is the set of 
positive integers. < 


Let R be the relation consisting of 5-tuples (A, N, S, D, T) representing airplane flights, 
where A is the airline, N is the flight number, S is the starting point, D is the destination, and T is 
the departure time. For instance, if Nadir Express Airlines has flight 963 from Newark to Bangor 


612 9/ Relations 


Links » 


at 15:00, then (Nadir, 963, Newark, Bangor, 15:00) belongs to R. The degree of this relation 
is 5, and its domains are the set of all airlines, the set of flight numbers, the set of cities, the set 
of cities (again), and the set of times. < 


9.2.3 Databases and Relations 


The time required to manipulate information in a database depends on how this information is 
stored. The operations of adding and deleting records, updating records, searching for records, 
and combining records from overlapping databases are performed millions of times each day ina 
large database. Because of the importance of these operations, various methods for representing 
databases have been developed. We will discuss one of these methods, called the relational data 
model, based on the concept of a relation. 

A database consists of records, which are n-tuples, made up of fields. The fields are the 
entries of the n-tuples. For instance, a database of student records may be made up of fields con- 
taining the name, student number, major, and grade point average of the student. The relational 
data model represents a database of records as an n-ary relation. Thus, student records are rep- 
resented as 4-tuples of the form (Student name, ID_number, Major, GPA). A sample database 
of six such records is 


(Ackermann, 231455, Computer Science, 3.88) 
(Adams, 888323, Physics, 3.45) 

(Chou, 102147, Computer Science, 3.49) 
(Goodfriend, 453876, Mathematics, 3.45) 
(Rao, 678543, Mathematics, 3.90) 

(Stevens, 786576, Psychology, 2.99). 


Relations used to represent databases are also called tables, because these relations are often 
displayed as tables. Each column of the table corresponds to an attribute of the database. For 
instance, the same database of students is displayed in Table 1. The attributes of this database 
are Student Name, ID Number, Major, and GPA. 

A domain of an n-ary relation is called a primary key when the value of the n-tuple from 
this domain determines the n-tuple. That is, a domain is a primary key when no two n-tuples in 
the relation have the same value from this domain. 

Records are often added to or deleted from databases. Because of this, the property that a 
domain is a primary key is time-dependent. Consequently, a primary key should be chosen that 
remains one whenever the database is changed. The current collection of n-tuples in a relation 
is called the extension of the relation. The more permanent part of a database, including the 
name and attributes of the database, is called its intension. When selecting a primary key, the 
goal should be to select a key that can serve as a primary key for all possible extensions of 
the database. To do this, it is necessary to examine the intension of the database to understand 
the set of possible n-tuples that can occur in an extension. 


TABLE 1 Students. 


Ackermann 231455 Computer Science 
Adams 888323 Physics 


Chou 102147 Computer Science 
Goodfriend 453876 Mathematics 

Rao 678543 Mathematics 
Stevens 786576 Psychology 
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Which domains are primary keys for the n-ary relation displayed in Table 1, assuming that no 
n-tuples will be added in the future? 


Solution: Because there is only one 4-tuple in this table for each student name, the domain 
of student names is a primary key. Similarly, the ID numbers in this table are unique, so the 
domain of ID numbers is also a primary key. However, the domain of major fields of study 
is not a primary key, because more than one 4-tuple contains the same major field of study. 
The domain of grade point averages is also not a primary key, because there are two 4-tuples 
containing the same GPA. < 


Combinations of domains can also uniquely identify n-tuples in an n-ary relation. When 
the values of a set of domains determine an n-tuple in a relation, the Cartesian product of these 
domains is called a composite key. 


Is the Cartesian product of the domain of major fields of study and the domain of GPAs a 
composite key for the n-ary relation from Table 1, assuming that no n-tuples are ever added? 


Solution: Because no two 4-tuples from this table have both the same major and the same GPA, 
this Cartesian product is a composite key. < 


Because primary and composite keys are used to identify records uniquely in a database, 
it is important that keys remain valid when new records are added to the database. Hence, 
checks should be made to ensure that every new record has values that are different in 
the appropriate field, or fields, from all other records in this table. For instance, it makes 
sense to use the student identification number as a key for student records if no two 
students ever have the same student identification number. A university should not use the name 
field as a key, because two students may have the same name (such as John Smith). 


9.2.4 Operations on n-ary Relations 


There are a variety of operations on n-ary relations that can be used to form new n-ary relations. 
Applied together, these operations can answer queries on databases that ask for all n-tuples that 
satisfy certain conditions. 

The most basic operation on an n-ary relation is determining all n-tuples in the n-ary relation 
that satisfy certain conditions. For example, we may want to find all the records of all computer 
science majors in a database of student records. We may want to find all students who have a 
grade point average above 3.5. We may want to find the records of all computer science majors 
who have a grade point average above 3.5. To perform such tasks we use the selection operator. 


Let R be an n-ary relation and C a condition that elements in R may satisfy. Then the selection 
operator sç maps the n-ary relation R to the n-ary relation of all n-tuples from R that satisfy 
the condition C. 


To find the records of computer science majors in the n-ary relation R shown in Table 1, we use 
the operator sç, where C} is the condition Major = “Computer Science.” The result is the two 4- 
tuples (Ackermann, 231455, Computer Science, 3.88) and (Chou, 102147, Computer Science, 
3.49). Similarly, to find the records of students who have a grade point average above 3.5 in 
this database, we use the operator Sc, where C, is the condition GPA > 3.5. The result is the 
two 4-tuples (Ackermann, 231455, Computer Science, 3.88) and (Rao, 678543, Mathematics, 
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TABLE 2 GPAs. 


Student_name 


Ackermann 
Adams 
Chou 
Goodfriend 
Rao 


Stevens 


a 


8 


9 


(0) 


GPA 


3.90). Finally, to find the records of computer science majors who have a GPA above 3.5, we 
use the operator sç,, where C; is the condition (Major = “Computer Science” A GPA > 3.5). 
The result consists of the single 4-tuple (Ackermann, 231455, Computer Science, 3.88). < 


Projections are used to form new n-ary relations by deleting the same fields in every record 
of the relation. 


maps the n-tuple (a,, a, ..., a„) to the 


m? 


The projection P; ; _,; where i; <i, <- <i 
TED aii ina ty 
m-tuple (a;,,4;,,...,4;,), where m < n. 


In other words, the projection P, ;,_; deletes n — m of the components of an n-tuple, leaving 


the i, th, ith, ..., and i,,th components. 


What results when the projection P,; is applied to the 4-tuples (2,3,0,4), (Jane 
Doe, 234111001, Geography, 3.14), and (a,, a>, a3, a4)? 


Solution: The projection P} , sends these 4-tuples to (2, 0), Jane Doe, Geography), and (a, a3), 
respectively. < 


Example 9 illustrates how new relations are produced using projections. 


What relation results when the projection P4 , is applied to the relation in Table 1? 


Solution: When the projection P} 4 is used, the second and third columns of the table are deleted, 
and pairs representing student names and grade point averages are obtained. Table 2 displays 
the results of this projection. < 


Fewer rows may result when a projection is applied to the table for a relation. This happens 
when some of the n-tuples in the relation have identical values in each of the m components of 
the projection, and only disagree in components deleted by the projection. For instance, consider 
the following example. 


What is the table obtained when the projection P, > is applied to the relation in Table 3? 


Solution: Table 4 displays the relation obtained when P; , is applied to Table 3. Note that there 
are fewer rows after this projection is applied. < 


TABLE 3 Enrollments. TABLE 4 Majors. 


Glauser Biology Glauser Biology 


Glauser Biology Marcus Mathematics 


Glauser Biology Miller Computer Science 


Marcus Mathematics 
Marcus Mathematics 
Marcus Mathematics 


Miller Computer Science 


Miller Computer Science 
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TABLE 5 Teaching_assignments. TABLE 6 Class_schedule. 


Course _ Course _ 
Professor Department number Department number Time 


Cruz Zoology Computer Science 2:00 P.M. 


Cruz Zoology Mathematics 3:00 P.M. 


Farber Psychology Mathematics 4:00 P.M. 


Farber Psychology Physics 4:00 P.M. 


Grammer Physics Psychology 3:00 P.M. 


Grammer Physics Psychology 11:00 A.M. 


Rosen Computer Science Zoology 9:00 A.M. 


Rosen Mathematics Zoology 8:00 A.M. 


Definition 4 


EXAMPLE 11 


The join operation is used to combine two tables into one when these tables share some 
identical fields. For instance, a table containing fields for airline, flight number, and gate, and 
another table containing fields for flight number, gate, and departure time can be combined into 
a table containing fields for airline, flight number, gate, and departure time. 


Let R be a relation of degree m and S a relation of degree n. The join JAR, S), 
where p<m and p<n, is a relation of degree m+n-—p that consists of all 


(m+n-—p)-tuples (dj, dp, donp Amp» Cf) C «+4 Cys b,, bo, a000 Apk where the m-tuple 
(Oio p aeo Oa ie Cp ooo Ca) belongs to R and the n-tuple (cy, co, sono Ep bi Day 000 Daas) 
belongs to S. 


In other words, the join operator J,, produces a new relation from two relations by combining all 
m-tuples of the first relation with all n-tuples of the second relation, where the last p components 
of the m-tuples agree with the first p components of the n-tuples. 


What relation results when the join operator J, is used to combine the relation displayed in 
Tables 5 and 6? 


Solution: The join J, produces the relation shown in Table 7. < 


There are other operators besides projections and joins that produce new relations from 
existing relations. A description of these operations can be found in books on database theory. 


TABLE 7 Teaching schedule. 


rotor | Dement | Cone nmi | tom [Tne 
335 


Cruz Zoology 9:00 A.M. 
Cruz Zoology 412 8:00 A.M. 
Farber Psychology 501 3:00 P.M. 
Farber Psychology 617 11:00 A.M. 
Grammer Physics 544 4:00 P.M. 


Rosen Computer Science 518 2:00 P.M. 
Rosen Mathematics 575 3:00 P.M. 
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TABLE 8 Flights. 


122 34 


Detroit 


Denver 


Anchorage 


Honolulu 
Detroit 
Denver 
Detroit 


9.2.5 SQL 


The database query language SQL (short for Structured Query Language) can be used to carry 
out the operations we have described in this section. Example 12 illustrates how SQL commands 
are related to operations on n-ary relations. 


We will illustrate how SQL is used to express queries by showing how SQL can be employed 
to make a query about airline flights using Table 8. The SQL statement 


SELECT Departure_time 
FROM Flights 
WHERE Destination=’ Detroit’ 


is used to find the projection P; (on the Departure_time attribute) of the selection of 5-tuples in 
the Flights database that satisfy the condition: Destination = ‘Detroit’. The output would be a 
list containing the times of flights that have Detroit as their destination, namely, 08:10, 08:47, 
and 09:44. SQL uses the FROM clause to identify the n-ary relation the query is applied to, the 
WHERE clause to specify the condition of the selection operation, and the SELECT clause to 
specify the projection operation that is to be applied. (Beware: SQL uses SELECT to represent 
a projection, rather than a selection operation. This is an unfortunate example of conflicting 
terminology.) < 


Example 13 shows how SQL queries can be made involving more than one table. 


The SQL statement 


SELECT Professor, Time 
FROM Teaching_assignments, Class_schedule 
WHERE Department=’Mathematics’ 


is used to find the projection P} ; of the 5-tuples in the database (shown in Table 7), which 
is the join J, of the Teaching_assignments and Class_schedule databases in Tables 5 and 6, 
respectively, which satisfy the condition: Department = Mathematics. The output would consist 
of the single 2-tuple (Rosen, 3:00 P.M.). The SQL FROM clause is used here to find the join of 
two different databases. < 


We have only touched on the basic concepts of relational databases in this section. More 
information can be found in [AhUI195]. 
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9.2.6 Association Rules from Data Mining 


We will now introduce concepts from data mining, the discipline with the goal of gaining 
useful information from data. In particular, we will discuss information that can be gleaned 
from databases of transactions. We will focus on supermarket transactions, but the ideas we 
present are relevant in a wide range of applications. 

By a transaction we mean a set of items bought by a customer during a visit to the store, 
such as {milk, eggs, bread} or {orange juice, bananas, yogurt, cream}. Stores collect large 
databases of transactions that can be used to help them manage their businesses. We will discuss 
how these databases can be used to address the question: How likely is it that a customer buys 
a product given that they also buy a collection of one or more specified products? 

We call each product in the store an item. A collection of items is known as an itemset. 
A k-itemset is an itemset that contains exactly k items. The terms transaction and basket are 
used synonymously with the word itemset. When a store has n items, a4, az, ..., d,, for sale, 
each transaction can be represented by an n-tuple b4, bz, ..., b,, where b; is a binary variable 
that tells us whether a; occurs in this transaction. That is, b; = 1 if a; is in this transaction and 
b; = 0 otherwise. (Note that we only care whether an item occurs in a transaction and not how 
many times it occurs.) We can represent a transaction by an ( + 1)-tuple of the form (trans- 
action number, b,, by, ..., b,,). The collection of all these (n + 1)-tuples forms a database of 
transactions. 

We now define additional terms used in the study of questions relating to the purchase of 
particular itemsets. 


The count of an itemset /, denoted by o(J), in a set of transactions T = {fj, ty, ..., tg}, where 
k is a positive integer, is the number of transactions that contain this itemset. That is, 


off) = |{t, ET | 1 Eny. 


The support of an itemset Z is the probability that Z is included in a randomly selected trans- 
action from T. That is, 


support(/) = = 


The support threshold s is specified for a particular application. Frequent itemset mining 
is the process of finding itemsets 7 with support greater than or equal to s. Such itemsets are 
said to be frequent. 


The morning transactions at a market stand that sells apples, pears, cider, donuts, and mangos are 
shown in Table 9. In Table 10 we display the corresponding binary database, where each record 
is an (n + 1)-tuple consisting of the transaction number followed by binary entries that represent 
this itemset. Because apples occurs in five of the eight transactions, we see that o({apples}) = 5 
and support({apples}) = 5/8. Similarly, because the itemset {apples, cider} is a subset of four 
of the eight transactions, we have o({apples, cider}) = 4 and support({cider}) = 4/8 = 1/2. 
If we set the support threshold to be 0.5, an item is frequent if it occurs in at least four 
of the eight transactions. Consequently, with this support threshold, apples, pears, donuts and 
cider are the frequent items. The itemset {apples, cider} is a frequent itemset, but the itemset 
{donuts pears} is not a frequent itemset. < 
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TABLE 9 A Set of Transactions. 


Transaction Number Items 


{apples, pears, mangos} 


{pears, cider} 


{apples, cider, donuts, mangos} 
{apples, pears, cider, donuts } 
{apples, cider, donuts} 

{pears, cider, donuts} 

{pears, donuts} 

{apples, pears, cider} 


TABLE 10 Binary Database for the Transactions in Table 9. 
Transaction Number Apples Cider Donuts Mangos 


1 
1 
0 
1 
0 
1 
1 
1 


We can use sets of transactions to help us predict whether a customer will buy a particular 
item given that they also buy all the items in an itemset (which might just be one item). Before 
we address a question of this type, we introduce some terminology. If S is a set of items and T is 
a set of transactions, an association rule is an implication of the form J > J, where I and J are 
disjoint subsets of S. Although this notation borrows the notation for logical implications, its 
meaning is not entirely analagous. The association rule 7 > J is not the statement that whenever 
I is a subset of a transaction, then J must also be one. Instead, the strength of an association rule 
is measured in terms of its support, the frequency of transactions that contain both J and J, and 
its confidence, the frequency of transactions that contain J when they also contain 7. 


Definition 6 IfI and J are subsets of a set T of transactions, then 


I 
support] > J) = — 
and 
oI UJ) 


confidence > J) = Sate 


The support of the association rule Z > J, the fraction of transactions that contain both / and J, 
is a useful measure because a low support value tells us that the basket containing all items in 


EXAMPLE 15 


Extra 
Examples 


Exercises 


1. List the triples in the relation {(a, b, c) | a, b, and c are 
integers withO <a<b<c<5}. 


2. Which 4-tuples are in the relation { (a, b, c, d) | a, b, c, and 6 
d are positive integers with abcd = 6}? 


3. List the 5-tuples in the relation in Table 8. 


» 
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I and all items in J is seldom purchased, whereas a high value tells us that they are purchased 
together in a large fraction of transactions. Note that the confidence of the association rule 1 > J 
is the conditional probability that a transaction will contain all the items in Z and in J given that 
it contains all the items in Z. So, the larger the confidence of J > J, the more likely it is for J to 
be a subset of a transaction that contains /. 


What are the support and the confidence of the association rule {cider, donuts} > {apples} for 
the set of transactions in Example 14? 


Solution: The support of this association rule is o({cider, donuts} U {apples})/|7|. Because 
o({cider, donuts} U {apples}) = o({cider, donuts, apples}) = 3 and |T| = 8, we see that the 
support of this rule is 3/8 = 0.375. 

The confidence of this rule is o({cider, donuts} U {apples})/o({cider, donuts}) = 3/4 = 
0.75. < 


An important problem in data mining is to find strong association rules, which have sup- 
port greater than or equal to a minimum support level and confidence greater than or equal to a 
minimum confidence level. It is important to have efficient algorithms to find strong association 
rules because the number of available items can be extremely large. For instance, a supermarket 
may have tens of thousands, or even hundreds of thousands, of items in stock. The brute-force 
approach of finding association rules with sufficiently large support and confidence by comput- 
ing the support and confidence of all possible association rules is infeasible because there are 
an exponential number of such association rules (see Exercise 41). Several widely used algo- 
rithms have been developed to solve this problem much more efficiently than brute force. Such 
algorithms first find frequent itemsets and then turn their attention to finding all the association 
rules with high confidence from the frequent itemsets that have been found. Consult data mining 
texts such as [Ag15] for details. 

Although we have presented association rules in the context of market baskets, they are 
useful in a surprisingly wide variety of applications. For instance, association rules can be used 
to improve medical diagnoses, in which itemsets are collections of test results or symptoms and 
transactions are the collections of test results and symptoms found on patient records. Associ- 
ation rules, in which itemsets are baskets of key words and transactions are the collections of 
words on web pages, are used by search engines. Cases of plagiarism can be found using as- 
sociation rules, in which itemsets are collections of sentences and transactions are the contents 
of documents. Association rules also play helpful roles in various aspects of computer security, 
including intrusion detection, in which the itemsets are collections of patterns and transactions 
are the strings transmitted during network attacks. The interested reader will be able to find 
many more such applications by searching the web. 


5. Assuming that no new n-tuples are added, find a compos- 
ite key with two fields containing the Airline field for the 
database in Table 8. 

. Assuming that no new n-tuples are added, find a compos- 
ite key with two fields containing the Professor field for 
the database in Table 7. 


7. The 3-tuples in a 3-ary relation represent the following at- 


. Assuming that no new n-tuples are added, find all the pri- 
mary keys for the relations displayed in 


a) Table 3. b) Table 5. 


c) Table 6. d) Table 8. 


tributes of a student database: student ID number, name, 
phone number. 

a) Is student ID number likely to be a primary key? 

b) Is name likely to be a primary key? 

c) Is phone number likely to be a primary key? 
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10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 
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The 4-tuples in a 4-ary relation represent these attributes 

of published books: title, ISBN, publication date, number 

of pages. 

a) What is a likely primary key for this relation? 

b) Under what conditions would (title, publication date) 
be a composite key? 

c) Under what conditions would (title, number of pages) 
be a composite key? 

The 5-tuples in a 5-ary relation represent these attributes 

of all people in the United States: name, Social Security 

number, street address, city, state. 

a) Determine a primary key for this relation. 

b) Under what conditions would (name, street address) 
be a composite key? 

c) Under what conditions would (name, street address, 
city) be a composite key? 

What do you obtain when you apply the selection oper- 

ator sç, Where C is the condition Room = A100, to the 

database in Table 7? 


What do you obtain when you apply the selection opera- 
tor sc, where C is the condition Destination = Detroit, to 
the database in Table 8? 


What do you obtain when you apply the selection 
operator sc, where C is the condition (Project =2)A 
(Quantity > 50), to the database in Table 10? 


What do you obtain when you apply the selection op- 
erator sç, where C is the condition (Airline = Nadir) V 
(Destination = Denver), to the database in Table 8? 


What do you obtain when you apply the projection P, 3 5 
to the 5-tuple (a, b, c, d, e)? 

Which projection mapping is used to delete the first, 
second, and fourth components of a 6-tuple? 

Display the table produced by applying the projection 
P14 to Table 8. 

Display the table produced by applying the projection 
P; 4 to Table 8. 

How many components are there in the n-tuples in the 
table obtained by applying the join operator J; to two 
tables with 5-tuples and 8-tuples, respectively? 


Construct the table obtained by applying the join 
operator J, to the relations in Tables 11 and 12. 


Show that if C} and C, are conditions that elements of the 
n-ary relation R may satisfy, then Sc, ac, ®) =Sc, (sc, ®)). 
Show that if C,; and C, are conditions that ele- 
ments of the n-ary relation R may satisfy, then 
Sc, (sc, (R)) = Sc, CSc, (R)). 

Show that if C is a condition that elements of 
the n-ary relations R and S may satisfy, then 
SC(R U S) = sc(R) U S(S). 

Show that if C is a condition that elements of 
the n-ary relations R and S may satisfy, then 
SCR NS) = sc(R)N S(S). 

Show that if C is a condition that elements of 
the n-ary relations R and S may satisfy, then 
SC(R — S) = s-(R) — s¢(S). 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


Show that if R and S are both n-ary relations, then 

Piinia RUS) = Pirinin O U Pirinin 0S): 

Give an example to show that if R and S are both n-ary 

relations, then P; ;,; (RAS) may be different from 

Pi ins esi, RO Pi. i,,..,4,, O) 

Give an example to show that if R and S are both n-ary 

relations, then P; ¿(R — S$) may be different from 
iiep ae ee A 

a) What are the operations that correspond to the query 

expressed using this SQL statement? 


SELECT Supplier 
FROM Part_needs 
WHERE 1000 < Part_number < 5000 


b) What is the output of this query given the database in 
Table 11 as input? 

a) What are the operations that correspond to the query 
expressed using this SQL statement? 


SELECT Supplier, Project 
FROM Part_needs, Parts_inventory 
WHERE Quantity < 10 


b) What is the output of this query given the databases 
in Tables 11 and 12 as input? 
Determine whether there is a primary key for the relation 
in Example 2. 
Determine whether there is a primary key for the relation 
in Example 3. 
Show that an n-ary relation with a primary key can be 
thought of as the graph of a function that maps values of 
the primary key to (n — 1)-tuples formed from values of 
the other domains. 
Suppose that the transactions at a convenience store dur- 
ing an evening are {bread, milk, diapers, juice}, {bread, 
milk, diapers, eggs}, {milk, diapers, beer, eggs}, {bread, 
beer}, {milk, diapers, eggs, juice}, and {milk, diapers, 
beer}. 
a) Find the count and support of diapers. 
b) Find all frequent itemsets if the threshold level is 0.6. 
Suppose that the key words on eight different web pages 
are {evolution, primate, Human, Neanderthal, DNA, 
fossil}, {evolution, Neanderthal, Denisovan, Human, 
DNA}, {cave, fossil, primate}, {Human, Neanderthal, 
Denisovan, evolution}, {DNA, genome, evolution, fos- 
sil}, {DNA, Human, Neanderthal, Denisovan, genome}, 
{evolution, primate, cave, fossil}, and {Human, Nean- 
derthal, genome}. 
a) Find the count and support of Neanderthal. 
b) Find all frequent itemsets if the threshold level is 0.6. 
Find the support and confidence of the association rule 
{beer} — {diapers} for the set of transactions in Exercise 
33. (This association rule has played an important role in 
the development of the subject.) 
Find the support and confidence of the association rule 
{human, DNA} > {Neanderthal} for the set of transac- 
tions in Exercise 34. 
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TABLE 11 Part_needs. TABLE 12 Parts_inventory. 


1 1 8 
3 1 2 
4 3 1 1 
3 2 2 
2 3 2 
4 4 1 
2 4 2 
1 2 4 


37. Suppose that J is an itemset with positive count in a set of b) Find the lift of the association rule {beer} > 
transactions. Find the confidence of the association rule {diapers} for the set of transactions in Exercises 33. 
I> 9. 


c) Find the lift of the association rule {evolution} > 
{Neanderthals, Denisovans} for the set of transac- 
tions in Exercise 34. 


38. Suppose that 7, J, and K are itemsets. Show that the six 
association rules {, J} > K, {, K} oI, {LK} >J, 
I> {J, K},J > {1 K},andK > {J, J} all have the same 
support. 

39. The lift of the association rule 7 — J, where J and J are 


40. Show that if an itemset is frequent in a set of transactions, 
then all its subsets are also frequent itemsets in this set 


itemsets with positive support in a set of transactions, nee an 

equals support(/ U J)/(support(/)support(/)). 41. Given n unique items, show that there are 3” possi- 

a) Show that the lift of J > J, when support(/) and ble association rules of the form J > J, where J and 
support(J/) are both positive, equals 1 if and only if the J are disjoint subsets of the set of all items. Be sure to 
occurrence of 7 in a transaction and the occurrence of allow the association rules where J or J, or both, are 
J in a transaction are independent events. empty. 


Representing Relations 


9.3.1 Introduction 


In this section, and in the remainder of this chapter, all relations we study will be binary relations. 
Because of this, in this section and in the rest of this chapter, the word relation will always refer 
to a binary relation. There are many ways to represent a relation between finite sets. As we have 
seen in Section 9.1, one way is to list its ordered pairs. Another way to represent a relation is to 
use a table, as we did in Example 3 in Section 9.1. In this section we will discuss two alternative 
methods for representing relations. One method uses zero—one matrices. The other method uses 
pictorial representations called directed graphs, which we will discuss later in this section. 

Generally, matrices are appropriate for the representation of relations in computer pro- 
grams. On the other hand, people often find the representation of relations using directed graphs 
useful for understanding the properties of these relations. 


9.3.2 Representing Relations Using Matrices 


A relation between finite sets can be represented using a zero—one matrix. Suppose that R is a 
relation from A = {q}, d, ...,d,,} to B = {b}, by, ..., b}. (Here the elements of the sets A and 
B have been listed in a particular, but arbitrary, order. Furthermore, when A = B we use the 
same ordering for A and B.) The relation R can be represented by the matrix Mp = [mj]. where 


_ flif@,b) ER, 
"y = 1 Oif (a, b) ER. 


l 
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EXAMPLE 1 


EXAMPLE 2 


FIGURE 1 The 
zero—one matrix 
for a reflexive 
relation. (Off 
diagonal ele- 
ments can 

be 0 or 1.) 


In other words, the zero—one matrix representing R has a 1 as its (i, j) entry when a, is related 
to b;, and a O in this position if a; is not related to b;. (Such a representation depends on the 
orderings used for A and B.) 

The use of matrices to represent relations is illustrated in Examples 1-6. 


Suppose that A = {1, 2,3} and B = {1, 2}. Let R be the relation from A to B containing (a, b) if 
a E€ A, b € B, and a > b. What is the matrix representing R if a, = 1, a, = 2, and a; = 3, and 
b; = 1 and b, = 2? 


Solution: Because R = {(2, 1), (3, 1), (3, 2)}, the matrix for R is 
0 0 
M =|1 O}. 
1 1 


The 1s in Mp show that the pairs (2, 1), (3, 1), and (3, 2) belong to R. The Os show that no other 
pairs belong to R. < 


Let A = {a}, dy, a3} and B = {b}, by, b3, b4, bs}. Which ordered pairs are in the relation R rep- 
resented by the matrix 


Solution: Because R consists of those ordered pairs (a; b;) with m; = 1, it follows that 


R = {(a,, by), (a, b,), (a, b3), (a, b4), (a3, b,), (az, b3), (a3, bs)}. < 


The matrix of a relation on a set, which is a square matrix, can be used to determine whether 
the relation has certain properties. Recall that a relation R on A is reflexive if (a, a) € R whenever 
a €A. Thus, R is reflexive if and only if (a;, a;) E R for i = 1,2, ..., n. Hence, R is reflexive if 
and only if m; = 1, for i = 1, 2, ..., n. In other words, R is reflexive if all the elements on the 
main diagonal of Me are equal to 1, as shown in Figure 1. Note that the elements off the main 
diagonal can be either 0 or 1. 

The relation R is symmetric if (a,b) €R implies that (b,a) € R. Consequently, the 
relation R on the set A = {a}, a5, ...,a,,} is symmetric if and only if (a; a;) E R whenever 
(a; a;) E R. In terms of the entries of Mg, R is symmetric if and only if m; = 1 whenever 
m;; = 1. This also means m;; = 0 whenever m, = 0. Consequently, R is symmetric if and only 
if M;i; = Mji for all pairs of integers i and j with į = 1, 2, ..., n andj = 1, 2, ... , n. Recalling the 
definition of the transpose of a matrix from Section 2.6, we see that R is symmetric if and only if 


Mr = Mpg”, 


that is, if Mp is a symmetric matrix. The form of the matrix for a symmetric relation is 
illustrated in Figure 2(a). 

The relation R is antisymmetric if and only if (a, b) E R and (b, a) E R imply that a = b. 
Consequently, the matrix of an antisymmetric relation has the property that if m, = 1 with i # j, 
then m,; = 0. Or, in other words, either m, = 0 or m; = 0 when i # j. The form of the matrix 
for an antisymmetric relation is illustrated in Figure 2(b). 


EXAMPLE 3 


EXAMPLE 4 
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(a) Symmetric (b) Antisymmetric 


FIGURE 2 The zero-one matrices for 
symmetric and antisymmetric relations. 


Suppose that the relation R on a set is represented by the matrix 


110 
M,=]1 1 1l. 
011 


Is R reflexive, symmetric, and/or antisymmetric? 


Solution: Because all the diagonal elements of this matrix are equal to 1, R is reflexive. More- 
over, because Mp is symmetric, it follows that R is symmetric. It is also easy to see that R is not 
antisymmetric. < 


The Boolean operations join and meet (discussed in Section 2.6) can be used to find the ma- 
trices representing the union and the intersection of two relations. Suppose that R, and R, are 
relations on a set A represented by the matrices Mg, and Ma, respectively. The matrix represent- 
ing the union of these relations has a 1 in the positions where either Mp or Mẹ, has a 1. The ma- 
trix representing the intersection of these relations has a 1 in the positions Ghee both Mg, and 
Ma, have a 1. Thus, the matrices representing the union and intersection of these relations are 


Maur, = Mr, V Me, and Mr nr, = Mr, ^ Mr,- 


Suppose that the relations R, and R, on a set A are represented by the matrices 


2 


1 0 1 1 0 1 
Mg} =|1 0 0 and Mp =|0 1 1j. 
0 1 0 1 0 0 
What are the matrices representing R, U R, and Rj NR,? 


Solution: The matrices of these relations are 


101 
Mee = MVM, =|1 1 1f, 

i 1-0 

101 
Maar, =Mp, A Mp, =|0 0 OJ. 


We now turn our attention to determining the matrix for the composite of relations. This 
matrix can be found using the Boolean product of the matrices (discussed in Section 2.6) 
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EXAMPLE 5 


EXAMPLE 6 


for these relations. In particular, suppose that R is a relation from A to B and S is a relation 
from B to C. Suppose that A, B, and C have m, n, and p elements, respectively. Let the zero— 
one matrices for S °R, R, and S be Ms op = [t;]. M} = [ri], and M, = [sil], respectively (these 
matrices have sizes m X p, m X n, and n X p, respectively). The ordered pair (a; c;) belongs to 
S OR if and only if there is an element b, such that (a;, b,) belongs to R and (b, c;) belongs to S. 
It follows that t;, = 1 if and only if r; = sọ = 1 for some k. From the definition of the Boolean 
product, this means that 


Myor = Mp © Ms. 


Find the matrix representing the relations S ° R, where the matrices representing R and S are 


Solution: The matrix for SR is 


111 
Mso =MpOMy=|0 1 1]. 
0 0 0 à 


The matrix representing the composite of two relations can be used to find the matrix 
for Mp.. In particular, 


— Mel 
Mpka — M, > 


from the definition of Boolean powers. Exercise 35 asks for a proof of this formula. 


Find the matrix representing the relation R?, where the matrix representing R is 
0 1 0 
Mrp=|0 1 Ij. 
1 0 0 


Solution: The matrix for R? is 


9.3.3 Representing Relations Using Digraphs 


We have shown that a relation can be represented by listing all of its ordered pairs or by using 
a zero—one matrix. There is another important way of representing a relation using a pictorial 
representation. Each element of the set is represented by a point, and each ordered pair is repre- 
sented using an arc with its direction indicated by an arrow. We use such pictorial representations 
when we think of relations on a finite set as directed graphs, or digraphs. 


Definition 1 


EXAMPLE 7 


a 
> 


d c 


FIGURE 3 
A directed graph. 


EXAMPLE 8 


EXAMPLE 9 


We will study directed 
graphs extensively in 
Chapter 10. 
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A directed graph, or digraph, consists of a set V of vertices (or nodes) together with a set 
E of ordered pairs of elements of V called edges (or arcs). The vertex a is called the initial 
vertex of the edge (a, b), and the vertex b is called the terminal vertex of this edge. 


An edge of the form (a, a) is represented using an arc from the vertex a back to itself. Such 
an edge is called a loop. 


The directed graph with vertices a, b, c, and d, and edges (a, b), (a, d), (b, b), (b, d), (c, a), (c, b), 
and (d, b) is displayed in Figure 3. < 


The relation R on a set A is represented by the directed graph that has the elements of A as its 
vertices and the ordered pairs (a, b), where (a, b) € R, as edges. This assignment sets up a one- 
to-one correspondence between the relations on a set A and the directed graphs with A as their 
set of vertices. Thus, every statement about relations corresponds to a statement about directed 
graphs, and vice versa. Directed graphs give a visual display of information about relations. As 
such, they are often used to study relations and their properties. (Note that relations from a set 
A to a set B can be represented by a directed graph where there is a vertex for each element 
of A and a vertex for each element of B, as shown in Section 9.1. However, when A = B, such 
representation provides much less insight than the digraph representations described here.) The 
use of directed graphs to represent relations on a set is illustrated in Examples 8-10. 


The directed graph of the relation 
R, = {(1, 1), (1, 3), (2, 1), 2 3), 2 4), G, 1), G, 2), (4, D} 


on the set {1, 2, 3, 4} is shown in Figure 4. <q 


What are the ordered pairs in the relation R, represented by the directed graph shown in 
Figure 5? 


Solution: The ordered pairs (x, y) in the relation are 
R, = {(1, 3), (1, 4}, (2 1), @ 2), 2, 3), B, 1), G, 3), (4, 1), (4, 3)}. 


Each of these pairs corresponds to an edge of the directed graph, with (2, 2) and (3, 3) corre- 
sponding to loops. 


The directed graph representing a relation can be used to determine whether the relation 
has various properties. For instance, a relation is reflexive if and only if there is a loop at ev- 
ery vertex of the directed graph, so that every ordered pair of the form (x, x) occurs in the 
relation. A relation is symmetric if and only if for every edge between distinct vertices in its 
digraph there is an edge in the opposite direction, so that (y, x) is in the relation whenever 
(x, y) is in the relation. Similarly, a relation is antisymmetric if and only if there are never two 
edges in opposite directions between distinct vertices. Finally, a relation is transitive if and only 
if whenever there is an edge from a vertex x to a vertex y and an edge from a vertex y to a 
vertex z, there is an edge from x to z (completing a triangle where each side is a directed edge 
with the correct direction). 


Remark: Note that a symmetric relation can be represented by an undirected graph, which is a 
graph where edges do not have directions. We will study undirected graphs in Chapter 10. 
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b c c d 


4 3 (a) Directed graph of S$; (b) Directed graph of S, 
FIGURE 4 The 
directed graph 


of the relation R}. 


FIGURE 5 The 
directed graph 
of the relation R,. 


FIGURE 6 The directed graphs of the 
relations S, and S,. 


EXAMPLE 10 Determine whether the relations for the directed graphs shown in Figure 6 are reflexive, sym- 
metric, antisymmetric, and/or transitive. 


Solution: Because there are loops at every vertex of the directed graph of S}, it is reflexive. The 
relation S} is neither symmetric nor antisymmetric because there is an edge from a to b but not 
one from b to a, but there are edges in both directions connecting b and c. Finally, S} is not 
transitive because there is an edge from a to b and an edge from b to c, but no edge from a to c. 

Because loops are not present at all the vertices of the directed graph of S,, this relation 
is not reflexive. It is symmetric and not antisymmetric, because every edge between distinct 
vertices is accompanied by an edge in the opposite direction. It is also not hard to see from the 
directed graph that S, is not transitive, because (c, a) and (a, b) belong to S}, but (c, b) does not 


belong to S$). < 
Exercises 
1. Represent each of these relations on {1, 2, 3} with a ma- 4. List the ordered pairs in the relations on {1, 2, 3, 4} corre- 
trix (with the elements of this set listed in increasing sponding to these matrices (where the rows and columns 
order). correspond to the integers listed in increasing order). 
a) {(1, 1), (1, 2), (1, 3)} 
b) {0 2), 2,1), 2 D, 8, 3} Toe Tre 
c) {C(L 1), (L 2), (1, 3), (2, 2), @ 3), G, 3)} a) b) 
d) {(1, 3), (3, 1)} O 1 1 1 00 1 1 
2. Represent each of these relations on {1,2,3,4} with a 1o11 EO 
matrix (with the elements of this set listed in increasing 
order). 0O 101 
a) {(1, 2), (1, 3), (1,4), (2, 3), 2, 4), (3,4) ie as 
b) {(1, 1), (1, 4), (2, 2), G, 3), (4, D} 1010 
© {(1, 2), (, 3), (d, 4), 2 D, @ 3) 2 4, G, D, 6, 2), 
(3, 4), (4, D, (4, 2), 4 3)} 5. How can the matrix representing a relation R on a set A 


d) {2,4} G, D, G, 2), G, 49) 

. List the ordered pairs in the relations on {1, 2, 3 } corre- 
sponding to these matrices (where the rows and columns 
correspond to the integers listed in increasing order). 


101 0 1 0 
a) |0 1 0 b)/0O 1 0 
101 0 1 0 
tL l 1 
cœ) |1 0 1 
t 1 1 


be used to determine whether the relation is irreflexive? 


. How can the matrix representing a relation R on a set A 


be used to determine whether the relation is asymmetric? 


. Determine whether the relations represented by the ma- 


trices in Exercise 3 are reflexive, irreflexive, symmetric, 
antisymmetric, and/or transitive. 


. Determine whether the relations represented by the ma- 


trices in Exercise 4 are reflexive, irreflexive, symmetric, 
antisymmetric, and/or transitive. 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


. How many nonzero entries does the matrix representing 


the relation R on A = {1, 2, 3, ..., 100} consisting of the 
first 100 positive integers have if R is 

a) {(a,b) |a >b}? b) {(a b) |a #b}? 

c) {(ab)|a=b+1}? d) {(a,b)|a=1}? 

e) {(a,b)|ab=1}? 

How many nonzero entries does the matrix representing 
the relation R on A = {1, 2, 3, ..., 1000} consisting of the 
first 1000 positive integers have if R is 

a) {(a b) |a <b}? 

b) {((4b)|a=b+1}? 

c) {(a,b)| a+ b = 1000}? 

d) {(a,b)|a+b < 1001}? 

e) {(a b) |a #0}? 

How can the matrix for R, the complement of the 
relation R, be found from the matrix representing R, 
when R is a relation on a finite set A? 

How can the matrix for R-!, the inverse of the 
relation R, be found from the matrix representing R, 
when R is a relation on a finite set A? 

Let R be the relation represented by the matrix 


011 
Me=|1 1 0l. 
101 


Find the matrix representing 

aR, b) R. c) R?. 

Let R, and R, be relations on a set A represented by the 
matrices 


0 10 0 10 
Mp, =|! 1 1] and Mp, =|0 1 1 
1 0 0 g 1 1 1 
Find the matrices that represent 
a) R, UR. b) Ri NR. c) RoR. 
d) R °R}. e) Ri OR. 


Let R be the relation represented by the matrix 


01 
M,=|0 0 
11 


oro 


Find the matrices that represent 

a) R. b) R°. c) Rt. 

Let R be a relation on a set A with n elements. If there 
are k nonzero entries in Mp, the matrix representing R, 
how many nonzero entries are there in Mp_., the matrix 
representing R!, the inverse of R? 

Let R be a relation on a set A with n elements. If there 
are k nonzero entries in Mp, the matrix representing R, 
how many nonzero entries are there in Mọ, the matrix 
representing R, the complement of R? 


Draw the directed graphs representing each of the rela- 
tions from Exercise 1. 


19. 


20. 


21. 


22. 
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Draw the directed graphs representing each of the rela- 
tions from Exercise 2. 
Draw the directed graph representing each of the rela- 
tions from Exercise 3. 
Draw the directed graph representing each of the rela- 
tions from Exercise 4. 


Draw the directed graph that represents the relation 
{(a, a), (a, b), (b, c), (c, b), (c, d), (d, a), (d, b)}. 


In Exercises 23—28 list the ordered pairs in the relations rep- 
resented by the directed graphs. 


23. 


25. 


27. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


a 


24. 
Q | 9 
b c 
26. 
a b 
c d | | 
Cc d 
28. 
a b 


é Oa QRO 


How can the directed graph of a relation R on a finite 
set A be used to determine whether a relation is asym- 
metric? 

How can the directed graph of a relation R on a finite 
set A be used to determine whether a relation is irreflex- 
ive? 

Determine whether the relations represented by the di- 
rected graphs shown in Exercises 23-25 are reflexive, ir- 
reflexive, symmetric, antisymmetric, and/or transitive. 
Determine whether the relations represented by the di- 
rected graphs shown in Exercises 26-28 are reflexive, ir- 
reflexive, symmetric, antisymmetric, asymmetric, and/or 
transitive. 

Let R be a relation on a set A. Explain how to use the di- 
rected graph representing R to obtain the directed graph 
representing the inverse relation R7!. 

Let R be a relation on a set A. Explain how to use the di- 
rected graph representing R to obtain the directed graph 
representing the complementary relation R. 

Show that if M} is the matrix representing the relation R, 
then Mu is the matrix representing the relation R”. 
Given the directed graphs representing two relations, 
how can the directed graph of the union, intersection, 
symmetric difference, difference, and composition of 
these relations be found? 
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9.4 Closures of Relations 


Definition 1 


EXAMPLE 1 


9.4.1 Introduction 


A computer network has data centers in Boston, Chicago, Denver, Detroit, New York, and San 
Diego. There are direct, one-way telephone lines from Boston to Chicago, from Boston to De- 
troit, from Chicago to Detroit, from Detroit to Denver, and from New York to San Diego. Let 
R be the relation containing (a, b) if there is a telephone line from the data center in a to that 
in b. How can we determine if there is some (possibly indirect) link composed of one or more 
telephone lines from one center to another? Because not all links are direct, such as the link from 
Boston to Denver that goes through Detroit, R cannot be used directly to answer this. In the lan- 
guage of relations, R is not transitive, so it does not contain all the pairs that can be linked. As 
we will show in this section, we can find all pairs of data centers that have a link by constructing 
a transitive relation S containing R such that S is a subset of every transitive relation containing 
R. Here, S is the smallest transitive relation that contains R. This relation is called the transitive 
closure of R. 


9.4.2 Different Types of Closures 


If R is a relation on a set A, it may or may not have some property P, such as reflexivity, symme- 
try, or transitivity. When R does not enjoy property P, we would like to find the smallest relation 
S on A with property P that contains R. 


If R is a relation on a set A, then the closure of R with respect to P, if it exists, is the relation 
S on A with property P that contains R and is a subset of every subset of A x A containing R 
with property P. 


If there is a relation S that is a subset of every relation containing R with property P, it must 
be unique. To see this, suppose that relations S and T both have property P and are subsets of 
every relation with property P that contains R. Then, S and T are subsets of each other, and so 
are equal. Such a relation, if it exists, is the smallest relation with property P that contains R 
because it is a subset of every relation with property P that contains R. 

We will show how reflexive, symmetric, and transitive closures of relations can be found. 
In Exercises 15 and 35 we give properties P for which the closure of a relation with respect to 
P may not exist. 

The relation R = {(1, 1), (1, 2), (2, 1), (3, 2)} on the set A = {1, 2, 3} is not reflexive. How 
can we produce a reflexive relation containing R that is as small as possible? This can be done 
by adding (2, 2) and (3, 3) to R, because these are the only pairs of the form (a, a) that are not in 
R. This new relation contains R. Furthermore, any reflexive relation that contains R must also 
contain (2, 2) and (3, 3). Because this relation contains R, is reflexive, and is contained within 
every reflexive relation that contains R, it is called the reflexive closure of R. 

As this example illustrates, given a relation R on a set A, the reflexive closure of R can be 
formed by adding to R all pairs of the form (a, a) with a € A, not already in R. The addition 
of these pairs produces a new relation that is reflexive, contains R, and is contained within 
any reflexive relation containing R. We see that the reflexive closure of R equals R U A, where 
A = {(a, a) | a € A} is the diagonal relation on A. (The reader should verify this.) 


What is the reflexive closure of the relation R = {(a, b) | a < b} on the set of integers? 
Solution: The reflexive closure of R is 


RUA={(ab)|a<b}U{(a4a|aEZ}={ab)|a<b}. < 


EXAMPLE 2 


Extra » 
Examples 


Definition 2 
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The relation {(1, 1), (1, 2), (2, 2), (2, 3), (3, 1), (3, 2)} on {1, 2, 3} is not symmetric. How can 
we produce a symmetric relation that is as small as possible and contains R? To do this, we need 
only add (2, 1) and (1, 3), because these are the only pairs of the form (b, a) with (a, b) € R that 
are not in R. This new relation is symmetric and contains R. Furthermore, any symmetric relation 
that contains R must contain this new relation, because a symmetric relation that contains R must 
contain (2, 1) and (1, 3). Consequently, this new relation is called the symmetric closure of R. 

As this example illustrates, the symmetric closure of a relation R can be constructed by 
adding all ordered pairs of the form (b, a), where (a, b) is in the relation, that are not already 
present in R. Adding these pairs produces a relation that is symmetric, that contains R, and 
that is contained in any symmetric relation that contains R. The symmetric closure of a re- 
lation can be constructed by taking the union of a relation with its inverse (defined in the 
preamble of Exercise 26 in Section 9.1); that is, R U R`! is the symmetric closure of R, where 
R! = {(b, a) | (a, b) € R}. The reader should verify this statement. 


What is the symmetric closure of the relation R = {(a, b) | a > b} on the set of positive integers? 


Solution: The symmetric closure of R is the relation 
RUR”! = {(a, b) | a > b} U {(b,a) |a > b} = {(a, b) | a £ b}. 


This last equality follows because R contains all ordered pairs of positive integers, where the 
first element is greater than the second element, and R7! contains all ordered pairs of positive 
integers, where the first element is less than the second. < 


Suppose that a relation R is not transitive. How can we produce a transitive relation that con- 
tains R such that this new relation is contained within any transitive relation that contains R? Can 
the transitive closure of a relation R be produced by adding all the pairs of the form (a, c), where 
(a, b) and (b, c) are already in the relation? Consider the relation R = {(1, 3), (1, 4), (2, 1), (3, 2)} 
on the set {1, 2, 3, 4}. This relation is not transitive because it does not contain all pairs of the 
form (a,c) where (a, b) and (b, c) are in R. The pairs of this form not in R are (1, 2), (2, 3), 
(2, 4), and (3, 1). Adding these pairs does not produce a transitive relation, because the result- 
ing relation contains (3, 1) and (1, 4) but does not contain (3, 4). This shows that constructing 
the transitive closure of a relation is more complicated than constructing either the reflexive 
or symmetric closure. The rest of this section develops algorithms for constructing transitive 
closures. As will be shown later in this section, the transitive closure of a relation can be found 
by adding new ordered pairs that must be present and then repeating this process until no new 
ordered pairs are needed. 


9.4.3 Paths in Directed Graphs 


We will see that representing relations by directed graphs helps in the construction of transitive 
closures. We now introduce some terminology that we will use for this purpose. 

A path in a directed graph is obtained by traversing along edges (in the same direction as 
indicated by the arrow on the edge). 


A path from a to b in the directed graph G is a sequence of edges (xo x1), (Xy, X2), 
(X>, X3), ..., (X,_1, X,) in G, where n is a nonnegative integer, and x) = a and x, = b, that 
is, a Sequence of edges where the terminal vertex of an edge is the same as the initial vertex 
in the next edge in the path. This path is denoted by x9, x1, X2 ..., X,_1, X, and has length n. 
We view the empty set of edges as a path of length zero from a to a. A path of length n > 1 
that begins and ends at the same vertex is called a circuit or cycle. 
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EXAMPLE 3 


THEOREM 1 


d e 


FIGURE 1 A directed graph. 


A path in a directed graph can pass through a vertex more than once. Moreover, an edge in 
a directed graph can occur more than once in a path. 


Which of the following are paths in the directed graph shown in Figure 1: a, b, e, d; a, e, c, d, b; 
b, a, c, b, a, a, b; d, c; c, b, a; e, b, a, b, a, b, e? What are the lengths of those that are paths? Which 
of the paths in this list are circuits? 


Solution: Because each of (a, b), (b, e), and (e, d) is an edge, a, b, e, d is a path of length three. 
Because (c, d) is not an edge, a, e, c, d, b is not a path. Also, b, a, c, b, a, a, b is a path of length 
six because (b, a), (a, c), (c, b), (b, a), (a, a), and (a, b) are all edges. We see that d, c is a path 
of length one, because (d, c) is an edge. Also c, b, a is a path of length two, because (c, b) and 
(b, a) are edges. All of (e, b), (b, a), (a, b), (b, a), (a, b), and (b, e) are edges, so e, b, a, b, a, b, e 
is a path of length six. 

The two paths b, a, c, b, a, a, b and e, b, a, b, a, b, e are circuits because they begin and end 
at the same vertex. The paths a, b, e, d; c, b, a; and d, c are not circuits. < 


The term path also applies to relations. Carrying over the definition from directed graphs to 
relations, there is a path from a to b in R if there is a sequence of elements a, x,,X,...,%,_1,0 


with (a, x1) E R, (x1, X2) E R,..., and (x,_), b) E R. Theorem 1 can be obtained from the defi- 
nition of a path in a relation. 


Let R be a relation on a set A. There is a path of length n, where n is a positive integer, from 
a to b if and only if (a, b) € R”. 


Proof: We will use mathematical induction. By definition, there is a path from a to b of length 
one if and only if (a, b) € R, so the theorem is true when n = 1. 

Assume that the theorem is true for the positive integer n. This is the inductive hypothesis. 
There is a path of length n + 1 from a to b if and only if there is an element c € A such that 
there is a path of length one from a to c, so (a, c) E R, and a path of length n from c to b, that 
is, (c, b) € R”. Consequently, by the inductive hypothesis, there is a path of length n + 1 from 
a to b if and only if there is an element c with (a, c) E R and (c, b) € R”. But there is such an 
element if and only if (a, b) € R"*!. Therefore, there is a path of length n + 1 from a to b if and 
only if (a, b) € R"*!. This completes the proof. < 


9.4.4 Transitive Closures 


We now show that finding the transitive closure of a relation is equivalent to determining which 
pairs of vertices in the associated directed graph are connected by a path. With this in mind, we 
define a new relation. 


Definition 3 


EXAMPLE 4 


EXAMPLE 5 


EXAMPLE 6 
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Let R be a relation on a set A. The connectivity relation R* consists of the pairs (a, b) such 
that there is a path of length at least one from a to b in R. 


Because R” consists of the pairs (a, b) such that there is a path of length n from a to b, it follows 
that R* is the union of all the sets R”. In other words, 


o0 


R* = Ur. 


n=1 


The connectivity relation is useful in many models. 


Let R be the relation on the set of all people in the world that contains (a, b) if a has met b. What 
is R”, where n is a positive integer greater than one? What is R*? 


Solution: The relation R? contains (a, b) if there is a person c such that (a, c) € R and (c, b) € R, 
that is, if there is a person c such that a has met c and c has met b. Similarly, R” consists 
of those pairs (a, b) such that there are people x4, Xz, -.., X„—-1 such that a has met x,,x, has 
met xz, ..., and x,_, has met b. 

The relation R* contains (a, b) if there is a sequence of people, starting with a and ending 
with b, such that each person in the sequence has met the next person in the sequence. (There 
are many interesting conjectures about R*. Do you think that this connectivity relation includes 
the pair with you as the first element and the president of Mongolia as the second element? We 
will use graphs to model this application in Chapter 10.) < 


Let R be the relation on the set of all subway stops in New York City that contains (a, b) if it is 
possible to travel from stop a to stop b without changing trains. What is R” when n is a positive 
integer? What is R*? 


Solution: The relation R” contains (a, b) if it is possible to travel from stop a to stop b by making 
at most n — 1 changes of trains. The relation R* consists of the ordered pairs (a, b) where it is 
possible to travel from stop a to stop b making as many changes of trains as necessary. (The 
reader should verify these statements.) < 


Let R be the relation on the set of all states in the United States that contains (a, b) if state a and 
state b have a common border. What is R”, where n is a positive integer? What is R*? 


Solution: The relation R” consists of the pairs (a, b), where it is possible to go from state a 
to state b by crossing exactly n state borders. R* consists of the ordered pairs (a, b), where it 
is possible to go from state a to state b crossing as many borders as necessary. (The reader 
should verify these statements.) The only ordered pairs not in R* are those containing states 
that are not connected to the continental United States (that is, those pairs containing Alaska or 
Hawaii). < 


Theorem 2 shows that the transitive closure of a relation and the associated connectivity 
relation are the same. 
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THEOREM 2 


LEMMA 1 


The transitive closure of a relation R equals the connectivity relation R*. 


Proof: Note that R* contains R by definition. To show that R* is the transitive closure of R 
we must also show that R* is transitive and that R* C S whenever S is a transitive relation that 
contains R. 

First, we show that R* is transitive. If (a, b) € R* and (b, c) € R*, then there are paths from 
a to b and from b to c in R. We obtain a path from a to c by starting with the path from a to b 
and following it with the path from b to c. Hence, (a, c) € R*. It follows that R* is transitive. 

Now suppose that S is a transitive relation containing R. Because S is transitive, $” also is 
transitive (the reader should verify this) and S” C S (by Theorem 1 of Section 9.1). Furthermore, 
because 


Sa Ù Sk 
k=1 


and S* C S, it follows that S* C S. Now note that if R C S, then R* C S*, because any 
path in R is also a path in S. Consequently, R* C S* C S. Thus, any transitive relation that 
contains R must also contain R*. Therefore, R* is the transitive closure of R. < 


Now that we know that the transitive closure equals the connectivity relation, we turn our 
attention to the problem of computing this relation. We do not need to examine arbitrarily long 
paths to determine whether there is a path between two vertices in a finite directed graph. As 
Lemma 1 shows, it is sufficient to examine paths containing no more than n edges, where n is 
the number of elements in the set. 


Let A be a set with n elements, and let R be a relation on A. If there is a path of length at least 
one in R from a to b, then there is such a path with length not exceeding n. Moreover, when 
a + b, if there is a path of length at least one in R from a to b, then there is such a path with 
length not exceeding n — 1. 


Proof: Suppose there is a path from a to b in R. Let m be the length of the shortest such path. 


Suppose that xo, X1, X2, -- -> Xm—1> Xp» Where xo = a and x,, = b, is such a path. 

Suppose that a = b and that m > n, so that m > n + 1. By the pigeonhole principle, be- 
cause there are n vertices in A, among the m vertices Xo, X1, ..., X,_1, at least two are equal (see 
Figure 2). 

X42 


ere > @ >. .-.->,.e > t=O ... oe | 
es Ki x2 X-i 454 Xy Xm=b 


FIGURE 2 Producing a path with length not exceeding n. 


THEOREM 3 


EXAMPLE 7 


Links > 
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Suppose that x; =x; with 0 <i<j<m-—1. Then the path contains a circuit from x; 
to itself. This circuit can be deleted from the path from a to b, leaving a path, namely, 


Xo Xp e+e Xp Nig -+ Xm—1 Xm from a to b of shorter length. Hence, the path of shortest length 
must have length less than or equal to n. 
The case where a # b is left as an exercise for the reader. < 
From Lemma 1, we see that the transitive closure of R is the union of R, R?, R?, ..., and R”. 


This follows because there is a path in R* between two vertices if and only if there is a path 
between these vertices in R’, for some positive integer i with i < n. Because 


R* =RUR2?UR UU R" 


and the zero—one matrix representing a union of relations is the join of the zero—one matrices of 
these relations, the zero—one matrix for the transitive closure is the join of the zero—one matrices 
of the first n powers of the zero—one matrix of R. 


Let M} be the zero—one matrix of the relation R on a set with n elements. Then the zero—one 
matrix of the transitive closure R* is 
= [2] [3] [n] 
Mr. = Meg YM; VM, Ve VM. 
Find the zero—one matrix of the transitive closure of the relation R where 
1 01 
M,p=|0 1 Oj}. 
1 1 0 
Solution: By Theorem 3, it follows that the zero—one matrix of R* is 


= [2] [3] 
Mr- = Mp V MÊ! v ME, 


Because 
1 1 1 1 1 1 
M? =|0 1 and Mẹ =j0 1 of, 
1 1 1 1 1 1 
it follows that 
10 1 1 1 1 1 1 1 1 1 1 
Mę-=ļ|0 1 Ojv|O 1 OjvjO 1 Oj=|0 1 O 
1 1 0 1 1 1 1 1 1 1 1 1 


< 


Theorem 3 can be used as a basis for an algorithm for computing the matrix of the 
relation R*. To find this matrix, the successive Boolean powers of Map, up to the nth power, are 
computed. As each power is calculated, its join with the join of all smaller powers is formed. 
When this is done with the nth power, the matrix for R* has been found. This procedure is 
displayed as Algorithm 1. 
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ALGORITHM 1 A Procedure for Computing the Transitive Closure. 


procedure transitive closure (Mp : zero—one n X n matrix) 
A:= Mp 
B:=A 
for i := 2 ton 
A:=AOM, 
B:=BvA 
return B{B is the zero—one matrix for R* } 


We can easily find the number of bit operations used by Algorithm 1 to determine the 
transitive closure of a relation. Computing the Boolean powers Ma, MU ey Mu requires that 
n—1 Boolean products of n Xn zero—one matrices be found. Each of these Boolean products 
can be found using n*(2n — 1) bit operations. Hence, these products can be computed using 
n?(2n — 1)(n— 1) bit operations. 

To find Mz. from the n Boolean powers of Mp, n—1 joins of zero—-one matrices need 
to be found. Computing each of these joins uses n? bit operations. Hence, (n — 1)n? bit op- 
erations are used in this part of the computation. Therefore, when Algorithm 1 is used, the 
matrix of the transitive closure of a relation on a set with n elements can be found using 
n?(2n — 1)(n— 1) + (n — 1)n? = 2n3(n — 1), which is O(n*) bit operations. The remainder of this 
section describes a more efficient algorithm for finding transitive closures. 


9.4.5 Warshall’s Algorithm 


Warshall’s algorithm, named after Stephen Warshall, who described this algorithm in 1960, 
is an efficient method for computing the transitive closure of a relation. Algorithm 1 can 
find the transitive closure of a relation on a set with n elements using 2n3(n— 1) bit opera- 
tions. However, the transitive closure can be found by Warshall’s algorithm using only 27° bit 
operations. 


Remark: Warshall’s algorithm is sometimes called the Roy—Warshall algorithm, because 
Bernard Roy described this algorithm in 1959. 


Suppose that R is a relation on a set with n elements. Let v,, v5, ..., v, be an arbitrary list- 
ing of these n elements. The concept of the interior vertices of a path is used in Warshall’s 
algorithm. If a, x,,X5,...,X,-,, b is a path, its interior vertices are x), X2, .-., Xm-1> that is, 
all the vertices of the path that occur somewhere other than as the first and last vertices 
in the path. For instance, the interior vertices of a path a, c, d, f, g, h, b,j in a directed graph 
are c, d, f, g, h, and b. The interior vertices of a, c, d, a, f, b are c, d, a, and f. (Note that the first 
vertex in the path is not an interior vertex unless it is visited again by the path, except as the 
last vertex. Similarly, the last vertex in the path is not an interior vertex unless it was visited 
previously by the path, except as the first vertex.) 

Warshall’s algorithm is based on the construction of a sequence of zero—one matrices. These 
matrices are Wo, W,, ..., W,,, where Wo = Mz is the zero—one matrix of this relation, and W, = 
[Ww], where wO = 1 if there is a path from v; to v, such that all the interior vertices of this path 
are in the set {v}, vz, ..., vg} (the first k vertices in the list) and is 0 otherwise. (The first and last 
vertices in the path may be outside the set of the first k vertices in the list.) Note that W, = Mp, 
because the (i, j)th entry of Mg. is 1 if and only if there is a path from v; to v,, with all interior 


n? 


EXAMPLE 8 


<a 
d 


FIGURE 3 
The directed 


graph of the 
relation R. 
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vertices in the set {v}, Vz, ..., V, } (but these are the only vertices in the directed graph). Example 
8 illustrates what the matrix W, represents. 


Let R be the relation with directed graph shown in Figure 3. Let a, b, c, d be a listing of the 
elements of the set. Find the matrices Wọ, W,, W3, W3, and W,. The matrix W, is the transitive 
closure of R. 


Solution: Let vy, = a, v, = b, v3 = c, and v, = d. Wọ is the matrix of the relation. Hence, 


W; has 1 as its (i, /)th entry if there is a path from v; to v, that has only v, = a as an interior 
vertex. Note that all paths of length one can still be used because they have no interior vertices. 
Also, there is now an allowable path from b to d, namely, b, a, d. Hence, 


W, has 1 as its (i, j)th entry if there is a path from v; to v; that has only v, = a and/or v, = b as 
its interior vertices, if any. Because there are no edges that have b as a terminal vertex, no new 
paths are obtained when we permit b to be an interior vertex. Hence, W, = W}. 

W; has 1 as its (7, j)th entry if there is a path from v; to v, that has only v; = a, v, = b, and/or 
v3 = c as its interior vertices, if any. We now have paths from d to a, namely, d, c, a, and from 
d to d, namely, d, c, d. Hence, 


0 0 


W, = 


ooco 


1 
1 1 1 
1 0 If 
1 1 1 
Finally, W, has 1 as its (7, j)th entry if there is a path from v, to v; that has v; = a, v, = b, 
v3 = c, and/or v, = d as interior vertices, if any. Because these are all the vertices of the graph, 
this entry is 1 if and only if there is a path from v; to v;. Hence, 


1 
1 
W,= 1 
1 


OOOD 


1 1 

1 1 

1 1 

1 1 

This last matrix, W,, is the matrix of the transitive closure. < 


Warshall’s algorithm computes Mp. by efficiently computing W,, W3, ..., W,, = Ma. This 
observation shows that we can compute W, directly from W,_,: There is a path from v; to v; 
with no vertices other than v}, vz, ..., v, as interior vertices if and only if either there is a path 
from v; to v; with its interior vertices among the first k — 1 vertices in the list, or there are paths 
from v, to v, and from v; to v, that have interior vertices only among the first k — 1 vertices in 
the list. That is, either a path from v, to v, already existed before v, was permitted as an interior 
vertex, or allowing v; as an interior vertex produces a path that goes from v; to v, and then from 
vz to v;. These two cases are shown in Figure 4. 
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LEMMA 2 


Case 1 


vi 


@_>—__e_>__e@ > o o -—»__ eo > 


ex 


All interior vertices 
in {vp vo.. Vet} 


. s . J 
i All interior vertices j 
in [VM vk-1} 


FIGURE 4 Adding v, to the set of allowable interior vertices. 


The first type of path exists if and only if wl -U1 — 1, and the second type of path exists if 


[1] [k-1] [k-1] ; 


and only if both wle and Wh is | or both 


whi 1 and we 4 


are 1. Hence, wit l is 1 if and only if either wi 


are 1. This om us Lemma 2. 


Let W, = [w; a be the zero—one matrix that has a 1 in its (7, /)th position if and only if there 
is a path from v; to v; with interior vertices from the set {v,, v,...,v,}. Then 


[k] _ ,, [k-1] [k-1] [k-1] 
wj = V (wi AW, ), 


whenever i, j, and k are positive integers not exceeding n. 


Lemma 2 gives us the means to compute efficiently the matrices W,, k = 1, 2, ..., n. We 
display the pseudocode for Warshall’s algorithm, using Lemma 2, as Algorithm 2. 


STEPHEN WARSHALL (1935-2006) Stephen Warshall, born in New York City, went to public school in 
Brooklyn. He attended Harvard University, receiving his degree in mathematics in 1956. He never received an 
advanced degree, because at that time no programs were available in his areas of interest. However, he took 
graduate courses at several different universities and contributed to the development of computer science and 
software engineering. 

After graduating from Harvard, Warshall worked at ORO (Operation Research Office), which was set up 
by Johns Hopkins to do research and development for the U.S. Army. In 1958 he left ORO to take a position 
at a company called Technical Operations, where he helped build a research and development laboratory for 


Courtesy of Stephen Warshall military software projects. In 1961 he left Technical Operations to found Massachusetts Computer Associates. 


Later, this company became part of Applied Data Research (ADR). After the merger, Warshall sat on the board 
of directors of ADR and managed a variety of projects and organizations. He retired from ADR in 1982. 


During his career Warshall carried out research and development in operating systems, compiler design, language design, and 
operations research. In the 1971-1972 academic year he presented lectures on software engineering at French universities. There is 
an interesting anecdote about his proof that the transitive closure algorithm, now known as Warshall’s algorithm, is correct. He and 
a colleague at Technical Operations bet a bottle of rum on who first could determine whether this algorithm always works. Warshall 
came up with his proof overnight, winning the bet and the rum, which he shared with the loser of the bet. Because Warshall did not 
like sitting at a desk, he did much of his creative work in unconventional places, such as on a sailboat in the Indian Ocean or in a 
Greek lemon orchard. 
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W := Me 
for k := 1 ton 
for i := 1 ton 
for j := 1 ton 


Wij = Wy V (Wig A Wg) 
return W{W = [wy] is Mp. } 


ALGORITHM 2 Warshall Algorithm. 


procedure Warshall (Mp : n X n zero—one matrix) 


The computational complexity of Warshall’s algorithm can easily be computed in terms of 


bit operations. To find the entry wl from the entries wl", w 


[k-1] | [k-1] [k-1] 


, and w; 


We using Lemma 


L L 
2 requires two bit operations. To find all n? entries of W, from those of W, , requires 2n? 
bit operations. Because Warshall’s algorithm begins with Wọ = Mp and computes the se- 
quence of n zero—one matrices W,, Wy, .. 


isn < 2n? = 20°. 


Exercises 


1. Let R be the relation on the set {0, 1, 2, 3} containing the 


ordered pairs (0, 1), (1, 1), (1, 2), (2, 0), (2, 2), and (3, 0). 
Find the 

a) reflexive closure of R. b) symmetric closure of R. 
. Let R be the relation {(a, b) | a # b} on the set of inte- 
gers. What is the reflexive closure of R? 

. Let R be the relation {(a, b) | a divides b} on the set of 
integers. What is the symmetric closure of R? 

. How can the directed graph representing the reflexive 
closure of a relation on a finite set be constructed from 
the directed graph of the relation? 


10. 


11. 


12. 


13. 


, W,, = Mp, the total number of bit operations used 


Find the smallest relation containing the relation in Ex- 
ample 2 that is both reflexive and symmetric. 

Find the directed graph of the smallest relation that is 
both reflexive and symmetric that contains each of the 
relations with directed graphs shown in Exercises 5-7. 
Suppose that the relation R on the finite set A is repre- 
sented by the matrix Mp. Show that the matrix that rep- 
resents the reflexive closure of R is Mp V L,- 

Suppose that the relation R on the finite set A is repre- 
sented by the matrix Mp. Show that the matrix that rep- 
resents the symmetric closure of R is Mg V Mọ- 


In Exercises 5—7 draw the directed graph of the reflexive clo- 14. Show that the closure of a relation R with respect to a 
sure of the relations with the directed graph shown. property P, if it exists, is the intersection of all the rela- 
5. 6. tions with property P that contain R. 


b 


O 
ao 


c d 


8. How can the directed graph representing the symmetric 


closure of a relation on a finite set be constructed from 
the directed graph for this relation? 


9. Find the directed graphs of the symmetric closures of the 


relations with directed graphs shown in Exercises 5-7. 


15. 


16. 


17. 


18. 


When is it possible to define the “irreflexive closure” 
of a relation R, that is, a relation that contains R, is ir- 
reflexive, and is contained in every irreflexive relation 
that contains R? 

Determine whether these sequences of vertices are paths 
in this directed graph. 
a) ab,ce 

b) b,e,c,b,e 

c) a,a,b,e,d, e 

d) b,c,e,d,a,a, b 

e) b,c,c,b,e,d, e, d 
f) a,a,b,b,c,c,b,e,d 


Find all circuits of length three in the directed graph in 
Exercise 16. 

Determine whether there is a path in the directed graph 
in Exercise 16 beginning at the first vertex given and 
ending at the second vertex given. 


a) a,b b) b,a c) b,b 
d) ae e) b,d f) cd 
g) d,d h) ea i) ec 
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19. 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


Let R be the relation on the set {1, 2, 3, 4, 5} containing 
the ordered pairs (1, 3), (2, 4), (3, 1), (3, 5), (4, 3), (5, 1), 


27. 


Use Warshall’s algorithm to find the transitive closures 
of the relations in Exercise 25. 


(5, 2), and (5, 4). Find 28. Use Warshall’s algorithm to find the transitive closures 
a) R?. b) RÈ. c) R*. of the relations in Exercise 26. 

5 6 * 
d) R. e) R°. f) R". 29. Find the smallest relation containing the relation 


Let R be the relation that contains the pair (a, b) if a and b 
are cities such that there is a direct nonstop airline flight 
from a to b. When is (a, b) in 


a) R°? b) R?? c) R*? 


Let R be the relation on the set of all students contain- 
ing the ordered pair (a, b) if a and b are in at least one 
common class and a # b. When is (a, b) in 


a) R?? b) R?? c) R*? 
Suppose that the relation R is reflexive. Show that R* is 
reflexive. 


Suppose that the relation R is symmetric. Show that R* 
is symmetric. 


30. 
31. 


{(1, 2), CL, 4), (3, 3), (4, 1)} that is 

a) reflexive and transitive. 

b) symmetric and transitive. 

c) reflexive, symmetric, and transitive. 

Finish the proof of the case when a # b in Lemma 1. 


Algorithms have been devised that use O(n**) bit op- 
erations to compute the Boolean product of two nxn 
zero—one matrices. Assuming that these algorithms can 
be used, give big-O estimates for the number of bit oper- 
ations using Algorithm 1 and using Warshall’s algorithm 
to find the transitive closure of a relation on a set with n 
elements. 


. . * 32. Devise an algorithm using the concept of interior vertices 
sles Mag ss the a eas R S irreflexive. Is the in a path to find the length of the shortest path between 
Telation dt" Necessarily 1TENeXIVe. two vertices in a directed graph, if such a path exists. 
Dee Algorithm 1 to find the transitive closures of these 33. Adapt Algorithm 1 to find the reflexive closure of the 
relations on {1, 2, 3, 4}. transitive closure of a relation on a set with n elements. 
x o ts A 2 n i a n 4,3)) 34. Adapt Warshall’s algorithm to find the reflexive closure 
© a i x i r 5 5 T n n of the transitive closure of a relation on a set with n 
d) {0 D, 0,4, 2 D, 2,3), BD, G, 2, 3,4), (4 2)) anon 
Use Algorithm 1 to find the transitive closures of these 35. Show that the closure with respect to the property P of 
relations on {a, b, c, d, e}. the relation R = {0, 0), 0, 1), (1, 1), (2,2)} on the set 
a) {(a, ©), (b, d, (c, a), (d, b), (e, d)} {0, 1, 2} does not exist if P is the property 
b) {(b, c), (b, e), (c, e), (d, a), (e, b), (e, €)} a) o not reflexive.” . 
©) {(a, b), (a, c), (a, €), (b, a), (b, c), (c, a), (c, b), (d, a), py Nas an odd number or elements. 

(e, d)} 36. Give an example of a relation R on the set {a, b, c} such 


d) {(a, e), (b, a), (b, d), (c, d), (d, a), (d, c), (e, a), (e, b), 
(e, c), (e, e)} 


Equivalence Relations 


9.5.1 Introduction 


that the symmetric closure of the reflexive closure of the 
transitive closure of R is not transitive. 


In some programming languages the names of variables can contain an unlimited number of 
characters. However, there is a limit on the number of characters that are checked when a com- 
piler determines whether two variables are equal. For instance, in traditional C, only the first 
eight characters of a variable name are checked by the compiler. (These characters are uppercase 
or lowercase letters, digits, or underscores.) Consequently, the compiler considers strings longer 
than eight characters that agree in their first eight characters the same. Let R be the relation on 
the set of strings of characters such that sRt, where s and ¢ are two strings, if s and f are at least 
eight characters long and the first eight characters of s and t agree, or s = t. It is easy to see that 
R is reflexive, symmetric, and transitive. Moreover, R divides the set of all strings into classes, 
where all strings in a particular class are considered the same by a compiler for traditional C. 
The integers a and b are related by the “congruence modulo 4” relation when 4 
divides a — b. We will show later that this relation is reflexive, symmetric, and transitive. It 
is not hard to see that a is related to b if and only if a and b have the same remainder when 
divided by 4. It follows that this relation splits the set of integers into four different classes. 


Links 


» 


Definition 1 


Equivalence relations 
are important in every 


branch of mathematics! 
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EXAMPLE 1 
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When we care only what remainder an integer leaves when it is divided by 4, we need only 
know which class it is in, not its particular value. 

These two relations, R and congruence modulo 4, are examples of equivalence relations, 
namely, relations that are reflexive, symmetric, and transitive. In this section we will show that 
such relations split sets into disjoint classes of equivalent elements. Equivalence relations arise 
whenever we care only whether an element of a set is in a certain class of elements, instead of 
caring about its particular identity. 


9.5.2 Equivalence Relations 


In this section we will study relations with a particular combination of properties that allows 
them to be used to relate objects that are similar in some way. 


A relation on a set A is called an equivalence relation if it is reflexive, symmetric, and 
transitive. 


Equivalence relations are important throughout mathematics and computer science. One 
reason for this is that in an equivalence relation, when two elements are related it makes sense 
to say they are equivalent. 


Two elements a and b that are related by an equivalence relation are called equivalent. The 
notation a ~ b is often used to denote that a and b are equivalent elements with respect to a 
particular equivalence relation. 


For the notion of equivalent elements to make sense, every element should be equivalent to 
itself, as the reflexive property guarantees for an equivalence relation. It makes sense to say 
that a and b are related (not just that a is related to b) by an equivalence relation, because 
when a is related to b, by the symmetric property, b is related to a. Furthermore, because an 
equivalence relation is transitive, if a and b are equivalent and b and c are equivalent, it follows 
that a and c are equivalent. 

Examples 1-5 illustrate the notion of an equivalence relation. 


Let R be the relation on the set of integers such that aRb if and only if a = b or a = —b. In 
Section 9.1 we showed that R is reflexive, symmetric, and transitive. It follows that R is an 
equivalence relation. < 


Let R be the relation on the set of real numbers such that aRb if and only if a — b is an integer. 
Is R an equivalence relation? 


Solution: Because a —a = 0 is an integer for all real numbers a, aRa for all real numbers a. 
Hence, R is reflexive. Now suppose that aRb. Then a — b is an integer, so b — a is also an inte- 
ger. Hence, bRa. It follows that R is symmetric. If aRb and bRc, then a — b and b — c are integers. 
Therefore, a — c = (a — b) + (b — c) is also an integer. Hence, aRc. Thus, R is transitive. Conse- 
quently, R is an equivalence relation. < 


One of the most widely used equivalence relations is congruence modulo m, where m is an 
integer greater than 1. 
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EXAMPLE 3 


EXAMPLE 4 


EXAMPLE 5 


EXAMPLE 6 


Congruence Modulo m Let m be an integer with m > 1. Show that the relation 
R= {(a, b) | a = b (mod m)} 


is an equivalence relation on the set of integers. 


Solution: Recall from Section 4.1 that a = b (mod m) if and only if m divides a — b. Note that 
a — a = Q is divisible by m, because 0 = 0 - m. Hence, a = a (mod m), so congruence modulo 
m is reflexive. Now suppose that a = b (mod m). Then a — b is divisible by m, so a — b = km, 
where k is an integer. It follows that b — a = (—k)m, so b = a (mod m). Hence, congruence 
modulo m is symmetric. Next, suppose that a = b (mod m) and b = c (mod m). Then m divides 
both a — b and b — c. Therefore, there are integers k and / with a — b = km and b — c = Im. 
Adding these two equations shows that a — c = (a — b) + (b — c) = km + Im = (k + I)m. Thus, 
a = c (mod m). Therefore, congruence modulo m is transitive. It follows that congruence modulo 
m is an equivalence relation. < 


Suppose that R is the relation on the set of strings of English letters such that aRb if and only if 
L(a) = I(b), where I(x) is the length of the string x. Is R an equivalence relation? 


Solution: Because l(a) = l(a), it follows that aRa whenever a is a string, so that R is re- 
flexive. Next, suppose that aRb, so that L(a) = l(b). Then bRa, because [(b) = l(a). Hence, 
R is symmetric. Finally, suppose that aRb and bRc. Then L(a) = I(b) and (b) = (c). Hence, 
l(a) = l(c), so aRc. Consequently, R is transitive. Because R is reflexive, symmetric, and tran- 
sitive, it is an equivalence relation. < 


Let n be a positive integer and S a set of strings. Suppose that R, is the relation on S such 
that sR,„t if and only if s = t, or both s and ¢ have at least n characters and the first n characters 
of s and t are the same. That is, a string of fewer than n characters is related only to itself; a 
string s with at least n characters is related to a string t if and only if t has at least n characters 
and t begins with the n characters at the start of s. For example, let n = 3 and let S be the set 
of all bit strings. Then sR,t either when s = f or both s and ż are bit strings of length 3 or more 
that begin with the same three bits. For instance, 01 R301 and 00111R;00101, but 01,010 and 
01011; 01110. 

Show that for every set S of strings and every positive integer n, R, is an equivalence relation 
on S. 


Solution: The relation R, is reflexive because s = s, so that sR,,s whenever s is a string in S. 
If sR,,t, then either s =¢ or s and ¢ are both at least n characters long that begin with the 
same n characters. This means that tR„s. We conclude that R, is symmetric. 

Now suppose that sk,,¢ and tR,u. Then either s = ¢ or s and t are at least n characters long 
and s and t begin with the same n characters, and either t = u or t and u are at least n characters 
long and ¢ and u begin with the same n characters. From this, we can deduce that either s = u or 
both s and u are n characters long and s and u begin with the same n characters (because in this 
case we know that s, t, and u are all at least n characters long and both s and u begin with the 
same n characters as ¢ does). Consequently, R,, is transitive. It follows that R, is an equivalence 
relation. < 


In Examples 6 and 7 we look at two relations that are not equivalence relations. 


Show that the “divides” relation on the set of positive integers in not an equivalence relation. 


Solution: By Examples 9 and 15 in Section 9.1, we know that the “divides” relation is reflex- 
ive and transitive. However, by Example 12 in Section 9.1, we know that this relation is not 
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EXAMPLE 8 
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symmetric (for instance, 2 | 4 but 4 ¥ 2). We conclude that the “divides” relation on the set of 
positive integers is not an equivalence relation. < 


Let R be the relation on the set of real numbers such that xRy if and only if x and y are real 
numbers that differ by less than 1, that is, |x — y| < 1. Show that R is not an equivalence relation. 


Solution: R is reflexive because |x — x| = 0 < 1 whenever x € R. R is symmetric, for if xRy, 
where x and y are real numbers, then |x — y| < 1, which tells us that |y — x| = |x — y| < 1, so 
that yRx. However, R is not an equivalence relation because it is not transitive. Take x = 2.8, 
y= 1.9, and z = 1.1, so that |x — y| = |2.8 — 1.9] = 0.9 < 1, |y- z| = |1.9-1.1] =0.8 < 1, 
but |x — z| = [2.8 — 1.1] = 1.7 > 1. That is, 2.8R 1.9, 1.9R1.1, but 2.8 Ř 1.1. < 


9.5.3 Equivalence Classes 


Let A be the set of all students in your school who graduated from high school. Consider the 
relation R on A that consists of all pairs (x, y), where x and y graduated from the same high 
school. Given a student x, we can form the set of all students equivalent to x with respect to R. 
This set consists of all students who graduated from the same high school as x did. This subset 
of A is called an equivalence class of the relation. 


Let R be an equivalence relation on a set A. The set of all elements that are related to an 
element a of A is called the equivalence class of a. The equivalence class of a with respect 
to R is denoted by [a]. When only one relation is under consideration, we can delete the 
subscript R and write [a] for this equivalence class. 


In other words, if R is an equivalence relation on a set A, the equivalence class of the element 
ais 

[a] = {s | (a, s) E R}. 
If b € [a]p, then b is called a representative of this equivalence class. Any element of a class 
can be used as a representative of this class. That is, there is nothing special about the particular 


element chosen as the representative of the class. 


What is the equivalence class of an integer for the equivalence relation of Example 1? 
Solution: Because an integer is equivalent to itself and its negative in this equivalence relation, 


it follows that [a] = {—a, a}. This set contains two distinct integers unless a = 0. For instance, 
[7] = {-7, 7}, [-5] = {-5, 5}, and [0] = {0}. < 


What are the equivalence classes of 0, 1, 2, and 3 for congruence modulo 4? 


Solution: The equivalence class of 0 contains all integers a such that a = 0 (mod 4). The integers 
in this class are those divisible by 4. Hence, the equivalence class of 0 for this relation is 


[0] = {...,-8, —4, 0, 4, 8, ... }. 
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EXAMPLE 10 


EXAMPLE 11 


The equivalence class of 1 contains all the integers a such that a = 1 (mod 4). The integers in 
this class are those that have a remainder of 1 when divided by 4. Hence, the equivalence class 
of 1 for this relation is 


[11 = Gis HT 8 SG 


The equivalence class of 2 contains all the integers a such that a = 2 (mod 4). The integers in 
this class are those that have a remainder of 2 when divided by 4. Hence, the equivalence class 
of 2 for this relation is 


[2] = {..., —6, —2, 2, 6, 10)... }. 


The equivalence class of 3 contains all the integers a such that a = 3 (mod 4). The integers in 
this class are those that have a remainder of 3 when divided by 4. Hence, the equivalence class 
of 3 for this relation is 


[3] = {..., -5, -1, 3, 7, 11, ... }. 


Note that every integer is in exactly one of the four equivalence classes and that the integer 
n is in the class containing n mod 4. < 


In Example 9 the equivalence classes of 0, 1, 2, and 3 with respect to congruence mod- 
ulo 4 were found. Example 9 can easily be generalized, replacing 4 with any positive integer 
m. The equivalence classes of the relation congruence modulo m are called the congruence 
classes modulo m. The congruence class of an integer a modulo m is denoted by [a],,, so 
[a],, = {..., a — 2m, a — m, a, a + m, a + 2m, ...}. For instance, from Example 9 we have [0], = 
{..., —8, —4, 0, 4, 8, ...}, [1]; = {..., —7, —3, 1, 5, 9, ...}, I2]; = {..., —6, —2, 2, 6, 10, ...}, and 
[3]; = {...,-5, 1, 3,7, 11, ...}. 


What is the equivalence class of the string 0111 with respect to the equivalence relation R} from 
Example 5 on the set of all bit strings? (Recall that sR, t if and only if s and ¢ are bit strings with 
s = t or s and t are strings of at least three bits that start with the same three bits.) 


Solution: The bit strings equivalent to 0111 are the bit strings with at least three bits that begin 
with 011. These are the bit strings 011, 0110, 0111, 01100, 01101, 01110, 01111, and so on. 
Consequently, 


[O11]p, = {011, 0110, 0111, 01100, 01101, 01110, 01111, ...}. < 


Identifiers in the C Programming Language In the C programming language, an identifier 
is the name of a variable, a function, or another type of entity. Each identifier is a nonempty 
string of characters where each character is a lowercase or an uppercase English letter, a digit, 
or an underscore, and the first character is a lowercase or an uppercase English letter. Identifiers 
can be any length. This allows developers to use as many characters as they want to name an 
entity, such as a variable. However, for compilers for some versions of C, there is a limit on the 
number of characters checked when two names are compared to see whether they refer to the 
same thing. For example, Standard C compilers consider two identifiers the same when they 
agree in their first 31 characters. Consequently, developers must be careful not to use identifiers 
with the same initial 31 characters for different things. We see that two identifiers are considered 
the same when they are related by the relation R}; in Example 5. Using Example 5, we know 
that R3,, on the set of all identifiers in Standard C, is an equivalence relation. 


THEOREM 1 
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What are the equivalence classes of each of the identifiers Number_of_tropical_ 
storms, Number_of_named-tropical_storms, and Number_of_named_tropical_storms_in_the_ 
Atlantic_in_2017? 


Solution: Note that when an identifier is less than 31 characters long, by the definition of R3,, 
its equivalence class contains only itself. Because the identifier Number_of_tropical_storms is 
25 characters long, its equivalence class contains exactly one element, namely, itself. 

The identifier Number_of_named_tropical_storms is exactly 31 characters long. An identi- 
fier is equivalent to it when it starts with these same 31 characters. Consequently, every identifier 
at least 31 characters long that starts with Number_of_named_tropical_storms is equivalent to 
this identifier. It follows that the equivalence class of Number_of_named_tropical_storms is the 
set of all identifiers that begin with the 31 characters Number_of_named _tropical_storms. 

An identifier is equivalent to the Number_of_named_tropical_storms_in_the_Atlantic_in_ 
2017 if and only if it begins with its first 31 characters. Because these characters 
are Number_of_named_tropical_storms, we see that an identifier is equivalent to Num- 
ber_of_named_tropical_storms_in_the_Atlantic_in_2017 if and only if it is equivalent to Num- 
ber_of_named_tropical_storms. It follows that these last two identifiers have the same equiva- 
lence class. < 


9.5.4 Equivalence Classes and Partitions 


Let A be the set of students at your school who are majoring in exactly one subject, and let R 
be the relation on A consisting of pairs (x, y), where x and y are students with the same major. 
Then R is an equivalence relation, as the reader should verify. We can see that R splits all stu- 
dents in A into a collection of disjoint subsets, where each subset contains students with a spec- 
ified major. For instance, one subset contains all students majoring (just) in computer science, 
and a second subset contains all students majoring in history. Furthermore, these subsets are 
equivalence classes of R. This example illustrates how the equivalence classes of an equiva- 
lence relation partition a set into disjoint, nonempty subsets. We will make these notions more 
precise in the following discussion. 

Let R be a relation on the set A. Theorem 1 shows that the equivalence classes of two 
elements of A are either identical or disjoint. 


Let R be an equivalence relation on a set A. These statements for elements a and b of A are 
equivalent: 


() aRb (ii) [a] =[b] (iii) [a] n [b] FB 


Proof: We first show that (i) implies (ii). Assume that aRb. We will prove that [a] = [b] by 
showing [a] C [b] and [b] € [a]. Suppose c € [a]. Then aRc. Because aRb and R is symmetric, 
we know that bRa. Furthermore, because R is transitive and bRa and aRc, it follows that bRc. 
Hence, c € [b]. This shows that [a] C [b]. The proof that [b] C [a] is similar; it is left as an 
exercise for the reader. 

Second, we will show that (ii) implies (iii). Assume that [a] = [b]. It follows that 
[a] N [b] # Ø because [a] is nonempty (because a € [a] because R is reflexive). 

Next, we will show that (iii) implies (i). Suppose that [a] N [b] #4 Ø. Then there is an 
element c with c € [a] and c € [b]. In other words, aRc and bRc. By the symmetric 
property, cRb. Then by transitivity, because aRc and cRb, we have aRb. 

Because (i) implies (ii), (ii) implies (iii), and (iii) implies (i), the three statements, (i), (ii), 
and (iii), are equivalent. 
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Recall that an index set 
is a set whose members 
label, or index, the 
elements of a set. 


EXAMPLE 12 


& 


FIGURE 1 A partition of a set. 


We are now in a position to show how an equivalence relation partitions a set. Let R be an 
equivalence relation on a set A. The union of the equivalence classes of R is all of A, because 
an element a of A is in its own equivalence class, namely, [a]. In other words, 


(iar =A. 
acA 


In addition, from Theorem 1, it follows that these equivalence classes are either equal or disjoint, 
so 


[aly N [bp = Ø, 


when [a]p # [b]p. 

These two observations show that the equivalence classes form a partition of A, because 
they split A into disjoint subsets. More precisely, a partition of a set S is a collection of disjoint 
nonempty subsets of § that have S as their union. In other words, the collection of subsets A,, 
i € I (where J is an index set) forms a partition of S if and only if 


A, #9 forie L 
A; NA; =Ø when i  j, 


and 


(Here the notation |),_, A; represents the union of the sets A; for all i € 7.) Figure 1 illustrates 
the concept of a partition of a set. 


Suppose that S = {1, 2, 3, 4,5,6}. The collection of sets A, = {1,2,3}, A, = {4,5}, and 
A, = {6} forms a partition of S, because these sets are disjoint and their union is S. < 


We have seen that the equivalence classes of an equivalence relation on a set form a parti- 
tion of the set. The subsets of S in this partition are the equivalence classes. Conversely, every 
partition of a set can be used to form an equivalence relation. Two elements are equivalent with 
respect to this relation if and only if they are in the same subset of S in the partition. 

To see this, assume that {A, | i € I} is a partition on S. Let R be the relation on S consisting 
of the pairs (x, y), where x and y belong to the same subset A; in the partition. To show that R is 
an equivalence relation we must show that R is reflexive, symmetric, and transitive. 

We see that (a, a) E R for every a E S, because a is in the same subset of S as itself. Hence, 
R is reflexive. If (a, b) € R, then b and a are in the same subset of S in the partition, so that 


THEOREM 2 


EXAMPLE 13 


EXAMPLE 14 


EXAMPLE 15 
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(b,a) E R as well. Hence, R is symmetric. If (a, b) E R and (b, c) E R, then a and b are in the 
same subset X of S in the partition, and b and c are in the same subset Y of S of the partition. 
Because the subsets of S in the partition are disjoint and b belongs to X and Y, it follows that 
X = Y. Consequently, a and c belong to the same subset of S in the partition, so (a, c) € R. Thus, 
R is transitive. 

It follows that R is an equivalence relation. The equivalence classes of R consist of subsets 
of S containing related elements, and by the definition of R, these are the subsets of S in the 
partition. Theorem 2 summarizes the connections we have established between equivalence 
relations and partitions. 


Let R be an equivalence relation on a set S. Then the equivalence classes of R form a partition 
of S. Conversely, given a partition {A; | i € I} of the set S, there is an equivalence relation R 
that has the sets A,, i € J, as its equivalence classes. 


Example 13 shows how to construct an equivalence relation from a partition. 


List the ordered pairs in the equivalence relation R produced by the partition A, = {1, 2, 3}, 
A, = {4,5}, and A; = {6} of S = {1, 2, 3, 4,5, 6}, given in Example 12. 


Solution: The subsets of S in the partition are the equivalence classes of R. The pair (a, b) € R if 
and only if a and b are in the same subset of the S in the partition. The pairs (1, 1), (1, 2), (1, 3), 
(2, 1), (2, 2), (2, 3), (3, 1), (3, 2), and (3, 3) belong to R because A, = {1, 2, 3} is an equivalence 
class; the pairs (4, 4), (4, 5), (5, 4), and (5, 5) belong to R because A, = {4, 5} is an equivalence 
class; and finally the pair (6, 6) belongs to R because {6} is an equivalence class. No pair other 
than those listed belongs to R. < 


The congruence classes modulo m provide a useful illustration of Theorem 2. There are m 
different congruence classes modulo m, corresponding to the m different remainders possible 
when an integer is divided by m. These m congruence classes are denoted by [0],,, [1], -> 
[m — 1],,. They form a partition of the set of integers. 


m’ 


What are the sets in the partition of the integers arising from congruence modulo 4? 


Solution: In Example 9 we found the four congruence classes, [O],, [1]4, [2]4, and [3],. They 
are the sets 


[0]; = {..., —8, —4, 0, 4, 8, ... } 
[1p = {...,—7, —3, 1, 5,9, coe J, 
[2], = {..., —6, —2, 2, 6, 10, ... }, 
[3]; = {...,-5,-1,3,7, 11... }. 


These congruence classes are disjoint, and every integer is in exactly one of them. In other 
words, as Theorem 2 says, these congruence classes form a partition. < 


We now provide an example of a partition of the set of all strings arising from an equivalence 
relation on this set. 


Let R; be the relation from Example 5. What are the sets in the partition of the set of all bit strings 
arising from the relation R, on the set of all bit strings? (Recall that sR, t, where s and ¢ are bit 
strings, if s = t or s and t are bit strings with at least three bits that agree in their first three bits.) 
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Solution: Note that every bit string of length less than three is equivalent only to itself. 
Hence [A]p, = {4}, [0]e, = {0}, [1p, = {1}, [00]g, = {00}, [01], = {01}, [10]g, = {10}, and 
[11], = {11}. Note that every bit string of length three or more is equivalent to one of the eight 
bit strings 000, 001, 010, 011, 100, 101, 110, and 111. We have 


[000]p, = {000, 0000, 0001, 00000, 00001, 00010, 00011, ...}, 


[001], = {001, 0010, 0011, 00100, 00101, 00110, 00111, ...}, 


[010]p, = {010, 0100, 0101, 01000, 01001, 01010, 01011, ...}, 


[011], = (011, 0110, 0111, 01100, 01101, 01110, 01111, ...}, 


[100]r, = {100, 1000, 1001, 10000, 10001, 10010, 10011, ...}, 


[101]p, = {101, 1010, 1011, 10100, 10101, 10110, 10111, ...}, 


[110], = {110, 1100, 1101, 11000, 11001, 11010, 11011, ...}, 


[111], = {111, 1110, 1111, 11100, 11101, 11110, 11111, ...}. 


These 15 equivalence classes are disjoint and every bit string is in exactly one of them. As 
Theorem 2 tells us, these equivalence classes partition the set of all bit strings. < 


Exercises 


1. Which of these relations on {0, 1, 2,3} are equivalence 


relations? Determine the properties of an equivalence re- 
lation that the others lack. 
a) {(0, 0), CL 1), ©, 2), 3, 3)} 
b) {(0, 0), ©, 2), (2, 0), (2, 2), (2, 3), G, 2), G, 3)} 
c) {(0,0), (L, 1), (1, 2), @, D, @, 2), 3, 3)} 
d) {(0, 0), (1, 1), (1, 3), @ 2), (2, 3), G, 1), G, 2), G, 3)} 
e) {(0, 0), (0, 1), ©, 2), C, 0), (1, 1), (1, 2), (2, 0), 

(2, 2), (3, 3)} 
. Which of these relations on the set of all people are equiv- 
alence relations? Determine the properties of an equiva- 
lence relation that the others lack. 


a) {(a, b) | a and bare the same age} 

b) {(a, b) | a and b have the same parents} 

c) {(a, b) | a and b share a common parent} 

d) {(a, b) | a and b have met} 

e) {(a, b) | a and b speak a common language } 

. Which of these relations on the set of all functions from Z 
to Z are equivalence relations? Determine the properties 
of an equivalence relation that the others lack. 


a (Fa) |fU)=s)} 

b) {0 8) | f@) = 80) or fC) = 801)} 

ce) (EDI fœ - ex) =1 forall x E Z} 

d) {fg)| for some CeEZ, for all xe Z f(x) - 
g(x) = C} 

e) (Gg) If) = g1) and f(1) = g(0)} 


10. 


. Define three equivalence relations on the set of students 


in your discrete mathematics class different from the re- 
lations discussed in the text. Determine the equivalence 
classes for each of these equivalence relations. 


. Define three equivalence relations on the set of buildings 


on a college campus. Determine the equivalence classes 
for each of these equivalence relations. 


. Define three equivalence relations on the set of classes of- 


fered at your school. Determine the equivalence classes 
for each of these equivalence relations. 


. Show that the relation of logical equivalence on the set 


of all compound propositions is an equivalence relation. 
What are the equivalence classes of F and of T? 


. Let R be the relation on the set of all sets of real numbers 


such that S RT if and only if S and T have the same car- 
dinality. Show that R is an equivalence relation. What are 
the equivalence classes of the sets {0, 1, 2} and Z? 


. Suppose that A is a nonempty set, and f is a function that 


has A as its domain. Let R be the relation on A consisting 
of all ordered pairs (x, y) such that f(x) = f(y). 


a) Show that R is an equivalence relation on A. 
b) What are the equivalence classes of R? 
Suppose that A is a nonempty set and R is an equivalence 


relation on A. Show that there is a function f with A as its 
domain such that (x, y) E€ R if and only if f(x) = fO). 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


Show that the relation R consisting of all pairs (x, y) such 
that x and y are bit strings of length three or more that 
agree in their first three bits is an equivalence relation on 
the set of all bit strings of length three or more. 


Show that the relation R consisting of all pairs (x, y) such 
that x and y are bit strings of length three or more that 
agree except perhaps in their first three bits is an equiva- 
lence relation on the set of all bit strings of length three 
or more. 


Show that the relation R consisting of all pairs (x, y) such 
that x and y are bit strings that agree in their first and third 
bits is an equivalence relation on the set of all bit strings 
of length three or more. 


Let R be the relation consisting of all pairs (x, y) such 
that x and y are strings of uppercase and lowercase En- 
glish letters with the property that for every positive in- 
teger n, the nth characters in x and y are the same letter, 
either uppercase or lowercase. Show that R is an equiva- 
lence relation. 


Let R be the relation on the set of ordered pairs 
of positive integers such that ((a, b), (c, d)) E R if and 
only if a+d = b+c. Show that R is an equivalence 
relation. 


Let R be the relation on the set of ordered pairs of pos- 
itive integers such that ((a, b), (c, d)) € R if and only if 
ad = bc. Show that R is an equivalence relation. 


(Requires calculus) 

a) Show that the relation R on the set of all differentiable 
functions from R to R consisting of all pairs (f, g) such 
that f’(x) = g'(x) for all real numbers x is an equiva- 
lence relation. 


b) Which functions are in the same equivalence class as 
the function f(x) = x”? 


(Requires calculus) 

a) Letn bea positive integer. Show that the relation R on 
the set of all polynomials with real-valued coefficients 
consisting of all pairs (f, g) such that f(x) = g(x) 
is an equivalence relation. [Here f(x) is the nth 
derivative of f(x).] 


b) Which functions are in the same equivalence class as 
the function f(x) = x+, where n = 3? 


Let R be the relation on the set of all URLs (or Web ad- 
dresses) such that x R y if and only if the Web page at x is 
the same as the Web page at y. Show that R is an equiva- 
lence relation. 


Let R be the relation on the set of all people who have 
visited a particular Web page such that x R y if and only 
if person x and person y have followed the same set of 
links starting at this Web page (going from Web page to 
Web page until they stop using the Web). Show that R is 
an equivalence relation. 
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In Exercises 21—23 determine whether the relation with the 
directed graph shown is an equivalence relation. 


21. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


22. 


a 
9 


d c 


Determine whether the relations represented by these 
zero—one matrices are equivalence relations. 


111 
a) {O11 b) 
111 


Show that the relation R on the set of all bit strings such 
that s Rt if and only if s and ż contain the same number 
of 1s is an equivalence relation. 


What are the equivalence classes of the equivalence rela- 
tions in Exercise 1? 


What are the equivalence classes of the equivalence rela- 
tions in Exercise 2? 


What are the equivalence classes of the equivalence rela- 
tions in Exercise 3? 


What is the equivalence class of the bit string 011 for the 
equivalence relation in Exercise 25? 


What are the equivalence classes of these bit strings for 
the equivalence relation in Exercise 11? 

a) 010 b) 1011 c) 11111 d) 01010101 
What are the equivalence classes of the bit strings 
in Exercise 30 for the equivalence relation from 
Exercise 12? 


What are the equivalence classes of the bit strings 
in Exercise 30 for the equivalence relation from 
Exercise 13? 


What are the equivalence classes of the bit strings in 
Exercise 30 for the equivalence relation R4 from Exam- 
ple 5 on the set of all bit strings? (Recall that bit strings s 
and f are equivalent under R, if and only if they are equal 
or they are both at least four bits long and agree in their 
first four bits.) 


What are the equivalence classes of the bit strings in Ex- 
ercise 30 for the equivalence relation R; from Example 5 
on the set of all bit strings? (Recall that bit strings s and 
t are equivalent under R; if and only if they are equal or 
they are both at least five bits long and agree in their first 
five bits.) 
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35. 


36. 


37. 


38. 


39. 


40. 


41. 


42. 


43. 


What is the congruence class [n], (that is, the equiva- 
lence class of n with respect to congruence modulo 5) 
when n is 

a) 2? b) 3? c) 6? d) —3? 
What is the congruence class [4],,, when m is 

a) 2? b) 3? c) 6? d) 8? 
Give a description of each of the congruence classes 
modulo 6. 


What is the equivalence class of each of these strings with 

respect to the equivalence relation in Exercise 14? 

a) No b) Yes c) Help 

a) What is the equivalence class of (1, 2) with respect to 
the equivalence relation in Exercise 15? 


b) Give an interpretation of the equivalence classes for 
the equivalence relation R in Exercise 15. [Hint: Look 
at the difference a — b corresponding to (a, b).] 


a) What is the equivalence class of (1, 2) with respect 
to the equivalence relation in Exercise 16? 


b) Give an interpretation of the equivalence classes for 
the equivalence relation R in Exercise 16. [Hint: Look 
at the ratio a/b corresponding to (a, b).] 


Which of these collections of subsets are partitions of 
{1, 2, 3,4, 5, 6}? 

a) {1,2}, {2,3,4}, {4 5, 6} 

b) {1}, (2,3, 6}, {4}, {5} 

c) {2,4,6}, (1,3, 5} d) {1,4,5}, {2,6} 

Which of these collections of subsets are partitions of 
{—3, —2, —1, 0, 1, 2,3}? 

a) {-3, —1, 1,3}, {—2, 0, 2} 

b) {—3, —2, —1, 0}, {0, 1, 2,3} 

c) {-3, 3}, {-2, 2}, {-L 1}, {0} 

d) {-3, -2, 2,3}, {-1, 1} 

Which of these collections of subsets are partitions of the 
set of bit strings of length 8? 

a) the set of bit strings that begin with 1, the set of bit 


strings that begin with 00, and the set of bit strings 
that begin with 01 


b) the set of bit strings that contain the string 00, the set 
of bit strings that contain the string 01, the set of bit 
strings that contain the string 10, and the set of bit 
strings that contain the string 11 


c) the set of bit strings that end with 00, the set of bit 
strings that end with 01, the set of bit strings that end 
with 10, and the set of bit strings that end with 11 


d) the set of bit strings that end with 111, the set of bit 
strings that end with 011, and the set of bit strings that 
end with 00 


e) the set of bit strings that contain 3k ones for some 
nonnegative integer k, the set of bit strings that con- 
tain 3k + 1 ones for some nonnegative integer k, and 
the set of bit strings that contain 3k + 2 ones for some 
nonnegative integer k. 


44. 


45. 


46. 


47. 


Which of these collections of subsets are partitions of the 

set of integers? 

a) the set of even integers and the set of odd integers 

b) the set of positive integers and the set of negative in- 
tegers 

c) the set of integers divisible by 3, the set of integers 
leaving a remainder of | when divided by 3, and the 
set of integers leaving a remainder of 2 when divided 
by 3 

d) the set of integers less than —100, the set of integers 
with absolute value not exceeding 100, and the set of 
integers greater than 100 

e) the set of integers not divisible by 3, the set of even 
integers, and the set of integers that leave a remainder 
of 3 when divided by 6 

Which of these are partitions of the set Z x Z of ordered 

pairs of integers? 

a) the set of pairs (x, y), where x or y is odd; the set of 
pairs (x, y), where x is even; and the set of pairs (x, y), 
where y is even 

b) the set of pairs (x, y), where both x and y are odd; the 
set of pairs (x, y), where exactly one of x and y is odd; 
and the set of pairs (x, y), where both x and y are even 

c) the set of pairs (x, y), where x is positive; the set of 
pairs (x, y), where y is positive; and the set of pairs 
(x, y), where both x and y are negative 

d) the set of pairs (x, y), where 3 | x and 3 | y; the set 
of pairs (x, y), where 3 | x and 3 J y; the set of pairs 
(x, y), where 3 f x and 3 | y; and the set of pairs (x, y), 
where 3 | x and3 ¥ y 

e) the set of pairs (x, y), where x > 0 and y > 0; the set 
of pairs (x, y), where x > 0 and y < 0; the set of pairs 
(x, y), where x < 0 and y > 0; and the set of pairs 
(x, y), where x < 0 and y < 0 

f) the set of pairs (x, y), where x 4 0 and y Æ 0; the set 
of pairs (x, y), where x = 0 and y # 0; and the set of 
pairs (x, y), where x # 0 and y = 0 

Which of these are partitions of the set of real numbers? 

a) the negative real numbers, {0}, the positive real 
numbers 

b) the set of irrational numbers, the set of rational 
numbers 


c) the set of intervals [k, k + 1], k = ..., —2, —1, 0, 
Dia 

d) the set of intervals (k, k + 1), k = ..., —2, —1, 0, 
2ra 

e) the set of intervals (k, k + 1], k = ..., —2, —1, 0, 
1,2, ... 


f) the sets {x+n |n € Z} for all x € [0, 1) 

List the ordered pairs in the equivalence relations pro- 
duced by these partitions of {0, 1, 2, 3, 4, 5}. 

a) {0}, {1 2}, {3,4,5} 

b) {0, 1}, {2,3}, {4,5} 

c) {0, 1, 2}, {3,4,5} 

d) {0}, {1}, {2}, {3}, {4}, {5} 


48. 


List the ordered pairs in the equivalence relations pro- 
duced by these partitions of {a, b, c, d, e, f, g}. 

a) {a,b}, {c,d}, fe fg} 

b) {a}, {b}, {c,d}, fe f}, {8} 

c) {a,b,c,d}, {e.f 8) 

d) {a,c e, 8}, {b, d}, {f} 


A partition P, is called a refinement of the partition P, if 
every set in P} is a subset of one of the sets in P}. 


49. 


50. 


51. 


Show that the partition formed from congruence classes 
modulo 6 is a refinement of the partition formed from 
congruence classes modulo 3. 


Show that the partition of the set of people living in the 
United States consisting of subsets of people living in the 
same county (or parish) and same state is a refinement of 
the partition consisting of subsets of people living in the 
same state. 


Show that the partition of the set of bit strings of length 16 
formed by equivalence classes of bit strings that agree on 
the last eight bits is a refinement of the partition formed 
from the equivalence classes of bit strings that agree on 
the last four bits. 


In Exercises 52 and 53, R, refers to the family of equivalence 
relations defined in Example 5. Recall that sR, t, where s 
and ¢ are two strings if s = t or s and ¢ are strings with at least 
n characters that agree in their first n characters. 


52. 


53. 


54. 


55. 


56. 


57. 


Show that the partition of the set of all bit strings formed 
by equivalence classes of bit strings with respect to the 
equivalence relation R, is a refinement of the partition 
formed by equivalence classes of bit strings with respect 
to the equivalence relation R}. 


Show that the partition of the set of all identifiers in C 
formed by the equivalence classes of identifiers with re- 
spect to the equivalence relation R3, is a refinement of 
the partition formed by equivalence classes of identi- 
fiers with respect to the equivalence relation Rg. (Com- 
pilers for “old” C consider identifiers the same when their 
names agree in their first eight characters, while compil- 
ers in standard C consider identifiers the same when their 
names agree in their first 31 characters.) 


Suppose that R, and R, are equivalence relations on a 
set A. Let P, and P, be the partitions that correspond to 
R, and R,, respectively. Show that R, C R, if and only if 
P, is a refinement of P}. 
Find the smallest equivalence relation on the set 
{a, b, c, d, e } containing the relation { (a, b), (a, c), (d, e)}. 
Suppose that R, and R, are equivalence relations on the 
set S. Determine whether each of these combinations 
of R, and R, must be an equivalence relation. 
a) R, UR, b) R, NR, c) RIOR, 
Consider the equivalence relation from Example 2, 
namely, R = {(x, y) | x — y is an integer}. 
a) What is the equivalence class of 1 for this equivalence 
relation? 


b) What is the equivalence class of 1/2 for this equiva- 
lence relation? 


* 58 


* 59 


60 


61 


62 


*63 


* 64 


65 


66 


67 
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. Each bead on a bracelet with three beads is either red, 
white, or blue, as illustrated in the figure shown. 


Bead 1 
Red 


Bead 3 
Blue 


Bead 2 
White 


Define the relation R between bracelets as: (B,, B3), 
where B, and B, are bracelets, belongs to R if and only 
if B, can be obtained from B} by rotating it or rotating it 
and then reflecting it. 


a) Show that R is an equivalence relation. 


b) What are the equivalence classes of R? 

. Let R be the relation on the set of all colorings of the 
2 x 2 checkerboard where each of the four squares is col- 
ored either red or blue so that (C;, C2), where C, and C, 
are 2 x 2 checkerboards with each of their four squares 
colored blue or red, belongs to R if and only if C, can be 
obtained from C, either by rotating the checkerboard or 
by rotating it and then reflecting it. 


a) Show that R is an equivalence relation. 


b) What are the equivalence classes of R? 

. a) Let R be the relation on the set of functions from Zt 
to Z* such that (f, g) belongs to R if and only if f 
is O(g) (see Section 3.2). Show that R is an equiva- 
lence relation. 


b) Describe the equivalence class containing f(n) = n? 
for the equivalence relation of part (a). 


. Determine the number of different equivalence relations 
on a set with three elements by listing them. 


. Determine the number of different equivalence relations 
on a set with four elements by listing them. 


. Do we necessarily get an equivalence relation when we 
form the transitive closure of the symmetric closure of 
the reflexive closure of a relation? 


. Do we necessarily get an equivalence relation when we 
form the symmetric closure of the reflexive closure of the 
transitive closure of a relation? 


. Suppose we use Theorem 2 to form a partition P from 
an equivalence relation R. What is the equivalence rela- 
tion R’ that results if we use Theorem 2 again to form an 
equivalence relation from P? 


. Suppose we use Theorem 2 to form an equivalence rela- 
tion R from a partition P. What is the partition P’ that 
results if we use Theorem 2 again to form a partition 
from R? 


. Devise an algorithm to find the smallest equivalence re- 
lation containing a given relation. 
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*68. Let p(n) denote the number of different equivalence Bell numbers after the American mathematician E. T. 
relations on a set with n elements (and by Theo- Bell.) 
rem 2 the number of partitions of a set with n ele- 
ments). Show that p(n) satisfies the recurrence relation 69. Use Exercise 68 to find the number of different equiv- 
pín) = pa C(n-— 1l,jp(n—j—1) and the initial alence relations on a set with n elements, where n is a 
condition p(0) = 1. (Note: The numbers p(n) are called positive integer not exceeding 10. 
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9.6.1 Introduction 


We often use relations to order some or all of the elements of sets. For instance, we order words 
using the relation containing pairs of words (x, y), where x comes before y in the dictionary. We 
schedule projects using the relation consisting of pairs (x, y), where x and y are tasks in a project 
such that x must be completed before y begins. We order the set of integers using the relation 
containing the pairs (x, y), where x is less than y. When we add all of the pairs of the form (x, x) 
to these relations, we obtain a relation that is reflexive, antisymmetric, and transitive. These are 
properties that characterize relations used to order the elements of sets. 


A relation R on a set S is called a partial ordering or partial order if it is reflexive, antisym- 
metric, and transitive. A set S together with a partial ordering R is called a partially ordered 
set, or poset, and is denoted by (S, R). Members of S are called elements of the poset. 


We give examples of posets in Examples 1-3. 


Show that the greater than or equal to relation (>) is a partial ordering on the set of integers. 


Solution: Because a > a for every integer a, > is reflexive. If a> b and b > a, then a = b. 
Hence, > is antisymmetric. Finally, > is transitive because a > b and b > c imply that a > c. It 


follows that > is a partial ordering on the set of integers and (Z, >) is a poset. < 


The divisibility relation | is a partial ordering on the set of positive integers, because it is 
reflexive, antisymmetric, and transitive, as was shown in Section 9.1. We see that (Zt, |) is a 
poset. Recall that (Z* denotes the set of positive integers.) < 


Show that the inclusion relation C is a partial ordering on the power set of a set S. 


Solution: Because A C A whenever A is a subset of S, C is reflexive. It is antisymmetric because 
A C Band B C A imply that A = B. Finally, C is transitive, because A C B and B C C imply that 


A C C. Hence, C is a partial ordering on P(S), and (P(S), C) is a poset. < 


Example 4 illustrates a relation that is not a partial ordering. 


Let R be the relation on the set of people such that xRy if x and y are people and x is older than 
y. Show that R is not a partial ordering. 


Solution: Note that R is antisymmetric because if a person x is older than a person y, then 
y is not older than x. That is, if xRy, then yRx. The relation R is transitive because if per- 
son x is older than person y and y is older than person z, then x is older than z. That is, if xRy 
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and yRz, then xRz. However, R is not reflexive, because no person is older than himself or 
herself. That is, xR x for all people x. It follows that R is not a partial ordering. < 


In different posets different symbols such as <, C, and |, are used for a partial ordering. 
However, we need a symbol that we can use when we discuss the ordering relation in an ar- 
bitrary poset. Customarily, the notation a x b is used to denote that (a, b) € R in an arbitrary 
poset (S, R). This notation is used because the less than or equal to relation on the set of real 
numbers is the most familiar example of a partial ordering and the symbol = is similar to the 
< symbol. (Note that the symbol = is used to denote the relation in any poset, not just the less 
than or equal to relation.) The notation a < b denotes that a x b, but a + b. Also, we say “a is 
less than b” or “b is greater than a” if a < b. 

When a and b are elements of the poset (S, <), it is not necessary that either a x b 
or b = a. For instance, in (P(Z), C), {1,2} is not related to {1,3}, and vice versa, because 
neither set is contained within the other. Similarly, in (Z*, |), 2 is not related to 3 and 3 is not 
related to 2, because 2 J 3 and 3 J 2. This leads to Definition 2. 


The elements a and b of a poset (S, x ) are called comparable if either a < b orb x a. When 
a and b are elements of S such that neither a x b nor b x a, a and bare called incomparable. 


In the poset (Z*, |), are the integers 3 and 9 comparable? Are 5 and 7 comparable? 
Solution: The integers 3 and 9 are comparable, because 3 | 9. The integers 5 and 7 are incom- 


parable, because 5 J 7 and7} 5. < 


The adjective “partial” is used to describe partial orderings because pairs of elements may 
be incomparable. When every two elements in the set are comparable, the relation is called a 
total ordering. 


If (S, x) is a poset and every two elements of S are comparable, S is called a totally ordered 
or linearly ordered set, and = is called a total order or a linear order. A totally ordered set 
is also called a chain. 


The poset (Z, <) is totally ordered, because a < b or b < a whenever a and b are integers. <4 


The poset (Z*, | ) is not totally ordered because it contains elements that are incomparable, such 
as 5 and 7. < 


In Chapter 6 we noted that (Z*, <) is well-ordered, where < is the usual less than or equal 
to relation. We now define well-ordered sets. 


(S, =<) is a well-ordered set if it is a poset such that = is a total ordering and every nonempty 
subset of S has a least element. 
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The set of ordered pairs of positive integers, Z*x Z*, with (a), a2) <(b,, b2) if a, < b}, or if 
a, = b, and a, < b, (the lexicographic ordering), is a well-ordered set. The verification of this 
is left as Exercise 53. The set Z, with the usual < ordering, is not well-ordered because the set 
of negative integers, which is a subset of Z, has no least element. < 


At the end of Section 5.3 we showed how to use the principle of well-ordered induction 
(there called generalized induction) to prove results about a well-ordered set. We now state and 
prove that this proof technique is valid. 


THE PRINCIPLE OF WELL-ORDERED INDUCTION Suppose that S is a well- 
ordered set. Then P(x) is true for all x € S, if 


INDUCTIVE STEP: For every y € S, if P(x) is true for all x € S with x < y, then P(y) is 
true. 


Proof: Suppose it is not the case that P(x) is true for all x € S. Then there is an element y € S 
such that P(y) is false. Consequently, the set A = {x € S | P(x) is false} is nonempty. Because S 
is well-ordered, A has a least element a. By the choice of a as a least element of A, we know 
that P(x) is true for all x € S with x < a. This implies by the inductive step P(a) is true. This 
contradiction shows that P(x) must be true for all x € S. 


Remark: We do not need a basis step in a proof using the principle of well-ordered induction 
because if xg is the least element of a well-ordered set, the inductive step tells us that P(x) is 
true. This follows because there are no elements x € S with x < x9, so we know (using a vacuous 
proof) that P(x) is true for all x € S with x < xp. 


The principle of well-ordered induction is a versatile technique for proving results about 
well-ordered sets. Even when it is possible to use mathematical induction for the set of positive 
integers to prove a theorem, it may be simpler to use the principle of well-ordered induction, as 
we saw in Examples 5 and 6 in Section 6.2, where we proved a result about the well-ordered set 
(NxN, =) where = is lexicographic ordering on N x N. 


9.6.2 Lexicographic Order 


The words in a dictionary are listed in alphabetic, or lexicographic, order, which is based on the 
ordering of the letters in the alphabet. This is a special case of an ordering of strings on a set 
constructed from a partial ordering on the set. We will show how this construction works in any 
poset. 

First, we will show how to construct a partial ordering on the Cartesian product of two 
posets, (A,, <,) and (A), x2). The lexicographic ordering =< on A, XA, is defined by spec- 
ifying that one pair is less than a second pair if the first entry of the first pair is less than 
(in A,) the first entry of the second pair, or if the first entries are equal, but the second en- 
try of this pair is less than (in A,) the second entry of the second pair. In other words, (a), a>) is 
less than (b,, b»), that is, 


(ay, a>) < (b,, by), 
either if a, <; b; or if both a, = b, and ay <, by. 


We obtain a partial ordering x by adding equality to the ordering < on A, X A). The veri- 
fication of this is left as an exercise. 
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FIGURE 1 The ordered pairs less than (3, 4) in lexicographic order. 


Determine whether (3, 5) < (4, 8), whether (3, 8) < (4,5), and whether (4, 9) < (4, 11) in the 
poset (Z x Z, x), where x is the lexicographic ordering constructed from the usual < relation 
on Z. 


Solution: Because 3 < 4, it follows that (3,5) < (4,8) and that (3, 8) < (4,5). We have 
(4, 9) < (4, 11), because the first entries of (4, 9) and (4, 11) are the same but 9 < 11. < 


In Figure 1 the ordered pairs in Z+x Z* that are less than (3,4) are highlighted. 
A lexicographic ordering can be defined on the Cartesian product of n posets (A,, x1), 
(Ay, <),-.-,(A,, Xn). Define the partial ordering < on A, X A, X = XA, by 


n? 


(Ay, do, ..-, Ay) < (Dy, bo, ..., Dp) 
if a, <,b,, or if there is an integer i > 0 such that a, = b}, ..., a; = b;, and a;,, <ii bipi In 
other words, one n-tuple is less than a second n-tuple if the entry of the first n-tuple in the first 
position where the two n-tuples disagree is less than the entry in that position in the second 
n-tuple. 


Note that (1, 2, 3, 5) < (1, 2, 4, 3), because the entries in the first two positions of these 4-tuples 
agree, but in the third position the entry in the first 4-tuple, 3, is less than that in the second 4- 
tuple, 4. (Here the ordering on 4-tuples is the lexicographic ordering that comes from the usual 
less than or equal to relation on the set of integers.) < 


We can now define lexicographic ordering of strings. Consider the strings a,a, ... a, and 
bb, ... b, ona partially ordered set S. Suppose these strings are not equal. Let ¢ be the minimum 
of m and n. The definition of lexicographic ordering is that the string aja, ...a,, is less than 
bb, ... b, if and only if 

(ai, Gy, ..., A,) < (dy, ba, ..., b,), or 

(ay, dy, ..., A) = (Hy, bo, ..., b,) andm <n, 
where < in this inequality represents the lexicographic ordering of S. In other words, to de- 
termine the ordering of two different strings, the longer string is truncated to the length of the 
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shorter string, namely, to t = min(m, n) terms. Then the t-tuples made up of the first ¢ terms of 
each string are compared using the lexicographic ordering on S’. One string is less than another 
string if the t-tuple corresponding to the first string is less than the t-tuple of the second string, 
or if these two f-tuples are the same, but the second string is longer. The verification that this is 
a partial ordering is left as Exercise 38 for the reader. 


Consider the set of strings of lowercase English letters. Using the ordering of letters in the 
alphabet, a lexicographic ordering on the set of strings can be constructed. A string is less than 
a second string if the letter in the first string in the first position where the strings differ comes 
before the letter in the second string in this position, or if the first string and the second string 
agree in all positions, but the second string has more letters. This ordering is the same as that 
used in dictionaries. For example, 


discreet < discrete, 

because these strings differ first in the seventh position, and e < t. Also, 
discreet < discreetness, 

because the first eight letters agree, but the second string is longer. Furthermore, 
discrete < discretion, 

because 


discrete < discreti. 


9.6.3 Hasse Diagrams 


Many edges in the directed graph for a finite poset do not have to be shown because they must be 
present. For instance, consider the directed graph for the partial ordering {(a, b) | a < b} on the 
set {1, 2, 3, 4}, shown in Figure 2(a). Because this relation is a partial ordering, it is reflexive, 
and its directed graph has loops at all vertices. Consequently, we do not have to show these loops 
because they must be present; in Figure 2(b) loops are not shown. Because a partial ordering is 
transitive, we do not have to show those edges that must be present because of transitivity. For 
example, in Figure 2(c) the edges (1, 3), (1, 4), and (2, 4) are not shown because they must be 
present. If we assume that all edges are pointed “upward” (as they are drawn in the figure), we 
do not have to show the directions of the edges; Figure 2(c) does not show directions. 

In general, we can represent a finite poset (S, x ) using this procedure: Start with the directed 
graph for this relation. Because a partial ordering is reflexive, a loop (a, a) is present at every 
vertex a. Remove these loops. Next, remove all edges that must be in the partial ordering because 


1 1 1 
(a) (b) (c) 


FIGURE 2 Constructing the Hasse diagram 
for ({1, 2, 3, 4}, <). 
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80 12 
4 6 
2 3 


(a) (b) (c) 
FIGURE 3 Constructing the Hasse diagram of ({1, 2, 3, 4, 6, 8, 12}, |). 


of the presence of other edges and transitivity. That is, remove all edges (x, y) for which there is 
an element z € S such that x < z and z < x. Finally, arrange each edge so that its initial vertex is 
below its terminal vertex (as it is drawn on paper). Remove all the arrows on the directed edges, 
because all edges point “upward” toward their terminal vertex. 

These steps are well defined, and only a finite number of steps need to be carried out for a 
finite poset. When all the steps have been taken, the resulting diagram contains sufficient infor- 
mation to find the partial ordering, as we will explain later. The resulting diagram is called the 
Hasse diagram of (S, <), named after the twentieth-century German mathematician Helmut 
Hasse who made extensive use of them. 

Let (S, x) be a poset. We say that an element y E€ S covers an element x € S if x < 
y and there is no element z € § such that x < z < y. The set of pairs (x,y) such that y 
covers x is called the covering relation of (S, x ). From the description of the Hasse diagram of 
a poset, we see that the edges in the Hasse diagram of (S,  ) are upwardly pointing edges cor- 
responding to the pairs in the covering relation of (S, =). Furthermore, we can recover a poset 
from its covering relation, because it is the reflexive transitive closure of its covering relation. 
(Exercise 31 asks for a proof of this fact.) This tells us that we can construct a partial ordering 
from its Hasse diagram. 


Draw the Hasse diagram representing the partial ordering {(a,b)|a divides b} on 
{1, 2, 3, 4, 6, 8, 12}. 


Solution: Begin with the digraph for this partial order, as shown in Figure 3(a). Remove all loops, 
as shown in Figure 3(b). Then delete all the edges implied by the transitive property. These 


HELMUT HASSE (1898-1979) Helmut Hasse was born in Kassel, Germany. He served in the German navy 
after high school. He began his university studies at Göttingen University in 1918, moving in 1920 to Marburg 
University to study under the number theorist Kurt Hensel. During this time, Hasse made fundamental con- 
tributions to algebraic number theory. He became Hensel’s successor at Marburg, later becoming director of 
the renowned mathematical institute at Gottingen in 1934, and took a position at Hamburg University in 1950. 
Hasse served for 50 years as an editor of Crelle’s Journal, an illustrious German mathematics periodical, taking 
over the job of chief editor in 1936 when the Nazis forced Hensel to resign. During World War II Hasse worked 
on applied mathematics research for the German navy. He was noted for the clarity and personal style of his lec- 
tures and was devoted both to number theory and to his students. (Hasse has been controversial for connections 
with the Nazi party. Investigations have shown he was a strong German nationalist but not an ardent Nazi.) 
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are (1, 4), (1, 6), (1, 8), (1, 12), (2, 8), (2, 12), and (3, 12). Arrange all edges to point upward, 
and delete all arrows to obtain the Hasse diagram. The resulting Hasse diagram is shown in 
Figure 3(c). < 


Draw the Hasse diagram for the partial ordering {(A, B) | A C B} on the power set P(S), where 
S = {a,b,c}. 


Solution: The Hasse diagram for this partial ordering is obtained from the associated digraph 
by deleting all the loops and all the edges that occur from transitivity, namely, (ø, {a, b}), 
(Ø, {a,c}), (Ø, {b,c}), (Ø, {a, b, c}), Cla}, {a b, c}), ({b}, {a, b, c}), and ({c}, {a, b, c}). Fi- 
nally, all edges point upward, and arrows are deleted. The resulting Hasse diagram is illustrated 
in Figure 4. < 


9.6.4 Maximal and Minimal Elements 


Elements of posets that have certain extremal properties are important for many applications. 
An element of a poset is called maximal if it is not less than any element of the poset. That is, a 
is maximal in the poset (S, =<) if there is no b € S such that a < b. Similarly, an element of a 
poset is called minimal if it is not greater than any element of the poset. That is, a is minimal 
if there is no element b € S such that b < a. Maximal and minimal elements are easy to spot 
using a Hasse diagram. They are the “top” and “bottom” elements in the diagram. 


Which elements of the poset ({2, 4, 5, 10, 12, 20, 25}, |) are maximal, and which are minimal? 


Solution: The Hasse diagram in Figure 5 for this poset shows that the maximal elements 
are 12, 20, and 25, and the minimal elements are 2 and 5. As this example shows, a poset 
can have more than one maximal element and more than one minimal element. < 


Sometimes there is an element in a poset that is greater than every other element. Such an 
element is called the greatest element. That is, a is the greatest element of the poset (S, =<) 
if b x a for all b € S. The greatest element is unique when it exists [see Exercise 40(a)]. Like- 
wise, an element is called the least element if it is less than all the other elements in the poset. 
That is, a is the least element of (S, x) if a x b for all b € S. The least element is unique when 
it exists [see Exercise 40(b)]. 


Determine whether the posets represented by each of the Hasse diagrams in Figure 6 have a 
greatest element and a least element. 


{a, b, c} 
12 20 
{a, c} {b, c} 
b 4 4 10 25 
{a} {b} 
Ø 2 5 
FIGURE 4 The Hasse diagram FIGURE 5 The Hasse 


of (P({a, b, c}), ©). diagram of a poset. 
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a a b a b a 


(a) (b) (c) (d) 
FIGURE 6 Hasse diagrams of four posets. 


Solution: The least element of the poset with Hasse diagram (a) is a. This poset has no greatest 
element. The poset with Hasse diagram (b) has neither a least nor a greatest element. The poset 
with Hasse diagram (c) has no least element. Its greatest element is d. The poset with Hasse 
diagram (d) has least element a and greatest element d. < 


Let S be a set. Determine whether there is a greatest element and a least element in the poset 
(P(S), ©). 


Solution: The least element is the empty set, because Ø C T for any subset T of S. The set S is 
the greatest element in this poset, because T C S whenever T is a subset of S. < 


Is there a greatest element and a least element in the poset (Z*, |)? 


Solution: The integer 1 is the least element because 1|n whenever n is a positive integer. 
Because there is no integer that is divisible by all positive integers, there is no greatest 
element. < 


Sometimes it is possible to find an element that is greater than or equal to all the elements 
in a subset A of a poset (S, =<). If u is an element of S such that a x u for all elements a € A, 
then u is called an upper bound of A. Likewise, there may be an element less than or equal 
to all the elements in A. If Z is an element of S such that / x a for all elements a € A, then / is 
called a lower bound of A. 


Find the lower and upper bounds of the subsets {a, b, c}, {j, h}, and {a, c, d, f } in the poset with 
the Hasse diagram shown in Figure 7. 


Solution: The upper bounds of {a, b, c} are e, f, j, and h, and its only lower bound is a. There 
are no upper bounds of {j, h}, and its lower bounds are a, b, c, d, e, and f. The upper bounds of 
{a, c,d, f} are f, h, and j, and its lower bound is a. < 


The element x is called the least upper bound of the subset A if x is an upper bound that 
is less than every other upper bound of A. Because there is only one such element, if it exists, 
it makes sense to call this element the least upper bound [see Exercise 42(a)]. That is, x is the 
least upper bound of A if a x x whenever a € A, and x = z whenever z is an upper bound of A. 
Similarly, the element y is called the greatest lower bound of A if y is a lower bound of A and 
z =< y whenever z is a lower bound of A. The greatest lower bound of A is unique if it exists [see 
Exercise 42(b)]. The greatest lower bound and least upper bound of a subset A are denoted by 
glb(A) and lub(A), respectively. 


Find the greatest lower bound and the least upper bound of {b, d, g}, if they exist, in the poset 
shown in Figure 7. 
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Solution: The upper bounds of {b, d, g} are g and h. Because g < h, g is the least upper bound. 
The lower bounds of {b, d, g} are a and b. Because a < b, b is the greatest lower bound. < 


Find the greatest lower bound and the least upper bound of the sets {3, 9, 12} and {1, 2, 4, 5, 10}, 
if they exist, in the poset (Z*, |). 


Solution: An integer is a lower bound of {3, 9, 12} if 3, 9, and 12 are divisible by this integer. 
The only such integers are 1 and 3. Because 1 | 3, 3 is the greatest lower bound of {3, 9, 12}. 
The only lower bound for the set {1, 2, 4,5, 10} with respect to | is the element 1. Hence, 1 is 
the greatest lower bound for {1, 2, 4, 5, 10}. 

An integer is an upper bound for {3, 9, 12} if and only if it is divisible by 3, 9, and 12. 
The integers with this property are those divisible by the least common multiple of 3, 9, and 
12, which is 36. Hence, 36 is the least upper bound of {3, 9, 12}. A positive integer is an upper 
bound for the set {1, 2, 4,5, 10} if and only if it is divisible by 1, 2, 4, 5, and 10. The integers 
with this property are those integers divisible by the least common multiple of these integers, 
which is 20. Hence, 20 is the least upper bound of {1, 2, 4, 5, 10}. < 


9.6.5 Lattices 


A partially ordered set in which every pair of elements has both a least upper bound and a 
greatest lower bound is called a lattice. Lattices have many special properties. Furthermore, 
lattices are used in many different applications such as models of information flow and play an 
important role in Boolean algebra. 


Determine whether the posets represented by each of the Hasse diagrams in Figure 8 are lattices. 


Solution: The posets represented by the Hasse diagrams in (a) and (c) are both lattices because 
in each poset every pair of elements has both a least upper bound and a greatest lower bound, 
as the reader should verify. On the other hand, the poset with the Hasse diagram shown in (b) 
is not a lattice, because the elements b and c have no least upper bound. To see this, note that 
each of the elements d, e, and f is an upper bound, but none of these three elements precedes 
the other two with respect to the ordering of this poset. < 


Is the poset (Z*, |) a lattice? 


Solution: Let a and b be two positive integers. The least upper bound and greatest lower bound 
of these two integers are the least common multiple and the greatest common divisor of these 


integers, respectively, as the reader should verify. It follows that this poset is a lattice. < 
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FIGURE 8 Hasse diagrams of three posets. 
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Determine whether the posets ({1, 2, 3, 4, 5}, |) and ({1, 2, 4, 8, 16}, |) are lattices. 


Solution: Because 2 and 3 have no upper bounds in ({1, 2, 3, 4, 5}, |), they certainly do not have 
a least upper bound. Hence, the first poset is not a lattice. 

Every two elements of the second poset have both a least upper bound and a greatest lower 
bound. The least upper bound of two elements in this poset is the larger of the elements and the 
greatest lower bound of two elements is the smaller of the elements, as the reader should verify. 
Hence, this second poset is a lattice. < 


Determine whether (P(S), C) is a lattice where S is a set. 


Solution: Let A and B be two subsets of S. The least upper bound and the greatest lower bound 
of A and B are AUB and A N B, respectively, as the reader can show. Hence, (P(S), C) is a 
lattice. < 


The Lattice Model of Information Flow In many settings the flow of information from one 
person or computer program to another is restricted via security clearances. We can use a lattice 
model to represent different information flow policies. For example, one common information 
flow policy is the multilevel security policy used in government and military systems. Each 
piece of information is assigned to a security class, and each security class is represented by a 
pair (A, C) where A is an authority level and C is a category. People and computer pro- 
grams are then allowed access to information from a specific restricted set of security 
classes. 

The typical authority levels used in the U.S. government are unclassified (0), confiden- 
tial (1), secret (2), and top secret (3). (Information is said to be classified if it is confidential, 
secret, or top secret.) Categories used in security classes are the subsets of a set of all compart- 
ments relevant to a particular area of interest. Each compartment represents a particular subject 
area. For example, if the set of compartments is {spies, moles, double agents}, then there are 
eight different categories, one for each of the eight subsets of the set of compartments, such as 
{spies, moles}. 

We can order security classes by specifying that (A4, C1) x (43, C2) if and only if 
A, <A, and C, C C,. Information is permitted to flow from security class (A,, C,) into 
security class (A), C3) if and only if (A,,C,) x (A, C2). For example, information is 
permitted to flow from the security class (secret, {spies, moles}) into the security class 
(top secret, {spies, moles, double agents}), whereas information is not allowed to flow from 
the security class (top secret, {spies, moles }) into either of the security classes (secret, {spies, 
moles, double agents}) or (top secret, {spies}). 

We leave it to the reader (see Exercise 48) to show that the set of all security classes with 
the ordering defined in this example forms a lattice. < 


9.6.6 Topological Sorting 


Suppose that a project is made up of 20 different tasks. Some tasks can be completed only after 
others have been finished. How can an order be found for these tasks? To model this problem 
we set up a partial order on the set of tasks so that a < b if and only if a and b are tasks where b 
cannot be started until a has been completed. To produce a schedule for the project, we need to 
produce an order for all 20 tasks that is compatible with this partial order. We will show how 
this can be done. 
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LEMMA 1 


EXAMPLE 26 


We begin with a definition. A total ordering x is said to be compatible with the partial 
ordering R if a x b whenever aRb. Constructing a compatible total ordering from a partial 
ordering is called topological sorting.* We will need to use Lemma 1. 


Every finite nonempty poset (S,  ) has at least one minimal element. 


Proof: Choose an element dy of S. If ag is not minimal, then there is an element a, with a, < do. 
If a, is not minimal, there is an element a, with a, < a,. Continue this process, so that if a, is 
not minimal, there is an element a,,, with a,,; < a„. Because there are only a finite number of 
elements in the poset, this process must end with a minimal element a,,. < 


The topological sorting algorithm we will describe works for any finite nonempty poset. 
To define a total ordering on the poset (A, =), first choose a minimal element a,; such an el- 
ement exists by Lemma 1. Next, note that (A — {a,}, =<) is also a poset, as the reader should 
verify. (Here by we mean the restriction of the original relation < on A to A — {a,}.) If it is 
nonempty, choose a minimal element a, of this poset. Then remove a, as well, and if there are 
additional elements left, choose a minimal element a3 in A — {a}, a, }. Continue this process by 
choosing a;,, to be a minimal element in A — {a}, dy, ..., a, }, as long as elements remain. 

Because A is a finite set, this process must terminate. The end product is a sequence of 
elements a, dy, ..., a„. The desired total ordering =, is defined by 

ai <i Ay Ke e Ki Gy. 
This total ordering is compatible with the original partial ordering. To see this, note that if b < c 
in the original partial ordering, c is chosen as the minimal element at a phase of the algorithm 
where b has already been removed, for otherwise c would not be a minimal element. Pseudocode 
for this topological sorting algorithm is shown in Algorithm 1. 


ALGORITHM 1 Topological Sorting. 


procedure topological sort ((S, =): finite poset) 


k:=1 

while S + Ø 
a, := a minimal element of S {such an element exists by Lemma 1} 
S := S — {a,} 
k:=k+1 

return 4), az, ..., A, {a1, do, ..., a, is a compatible total ordering of S} 


Find a compatible total ordering for the poset ({1, 2, 4, 5, 12, 20}, |). 


Solution: The first step is to choose a minimal element. This must be 1, because it is the only 
minimal element. Next, select a minimal element of ({2, 4, 5, 12, 20}, |). There are two minimal 
elements in this poset, namely, 2 and 5. We select 5. The remaining elements are {2, 4, 12, 20}. 
The only minimal element at this stage is 2. Next, 4 is chosen because it is the only minimal 


*“Topological sorting” is terminology used by computer scientists; mathematicians use the terminology “linearization of a 
partial ordering” for the same thing. In mathematics, topology is the branch of geometry dealing with properties of geometric 
figures that hold for all figures that can be transformed into one another by continuous bijections. In computer science, a 
topology is any arrangement of objects that can be connected with edges. 


EXAMPLE 27 


A Cc E 


FIGURE 10 The 
Hasse diagram for 
seven tasks. 
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Minimal 
element 1 5 2 4 20 12 
chosen 


FIGURE 9 A topological sort of ({1, 2, 4, 5, 12, 20}, |). 


element of ({4, 12, 20}, |). Because both 12 and 20 are minimal elements of ({ 12, 20}, |), either 
can be chosen next. We select 20, which leaves 12 as the last element left. This produces the 
total ordering 


1<5 <2 «4 < 20 < 12. 


The steps used by this sorting algorithm are displayed in Figure 9. < 


Topological sorting has an application to the scheduling of projects. 


A development project at a computer company requires the completion of seven tasks. Some of 
these tasks can be started only after other tasks are finished. A partial ordering on tasks is set up 
by considering task X < task Y if task Y cannot be started until task X has been completed. The 
Hasse diagram for the seven tasks, with respect to this partial ordering, is shown in Figure 10. 
Find an order in which these tasks can be carried out to complete the project. 


Solution: An ordering of the seven tasks can be obtained by performing a topolog- 
ical sort. The steps of a sort are illustrated in Figure 11. The result of this sort, 


A <C < B<E < F < D < G, gives one possible order for the tasks. < 
G G G G G G z 
D F D F D, F | D F o/ Ne »/ 
B B B 
A C E C E E E 
Minimal 
element A C B E F D G 
chosen 


FIGURE 11 A topological sort of the tasks. 
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Exercises 


. Which of these relations on {0, 1, 2, 3} are partial order- 
ings? Determine the properties of a partial ordering that 
the others lack. 


a) {(0, 0), (1, 1), @, 2), 3, 3)} 

b) {(0, 0), CL 1), @ 0), (2, 2), (2, 3), G, 2), G, 3)} 
c) {(0,0), CL 1), CL, 2), 2, 2), GB, 3)} 

d) {(0,0), (1, 1), CL, 2), CL 3), (2, 2), (2, 3), G, 3)} 


e) {(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), 

(2, 2), (3, 3)} 
. Which of these relations on {0, 1, 2,3} are partial order- 
ings? Determine the properties of a partial ordering that 
the others lack. 


a) {(0, 0), (2, 2), (3, 3)} 

b) {(0,0), (1, 1), (2, 0), (2, 2), 2, 3), (3, 3)} 

c) {(0,0), CL 1), C, 2), (2, 2), G, D, 3, 3)} 

d) {(0,0), (1, 1), G, 2), (1, 3), (2, 0), (2, 2), (2, 3), 
(3, 0), (3, 3)} 

e) {(0, 0), (0, 1), (0, 2), (0, 3), (1, 0), (1, 1), (1, 2), 
(1, 3), (2, 0), (2, 2), (3, 3)} 

. Is (S, R) a poset if S is the set of all people in the world 

and (a, b) E€ R, where a and b are people, if 

a) ais taller than b? 

b) ais not taller than b? 

c) a =b or ais an ancestor of b? 

d) a and b have a common friend? 


. Is (S, R) a poset if S is the set of all people in the world 
and (a, b) E R, where a and b are people, if 


a) ais no shorter than b? 

b) a weighs more than b? 

c) a= b or ais a descendant of b? 

d) a and b do not have a common friend? 

. Which of these are posets? 

a) (Z=) b) ZF) O42) dd ZY) 

. Which of these are posets? 

a R=) b R<) ORS) d RF) 

. Determine whether the relations represented by these 
zero—one matrices are partial orders. 


111 iti 
alı 1.0 b) [0 1 0 
001 001 
1110 
01 1 0 
910 011 
1101 


. Determine whether the relations represented by these 
zero—one matrices are partial orders. 


1 0 1 1 0 0 
a) |i 1 0 b)jO 1 0 
0 0 1 1 0 1 
1 0 1 0 
gjo LLO 
001 1 
1 1 0 1 


In Exercises 9—1 1 determine whether the relation with the di- 
rected graph shown is a partial order. 


9. 10. 


11. 


Oa) 


12. Let (S, R) be a poset. Show that (S, R`!) is also a poset, 
where R7! is the inverse of R. The poset ($, R7!) is called 
the dual of (S, R). 

13. Find the duals of these posets. 
a) ({0, 1,2}, <) b) (Z, >) 
c) (P(Z), 2) d) (Z*, |) 

14. Which of these pairs of elements are comparable in the 
poset (Z*, |)? 


a) 5,15 b) 6,9 c) 8, 16 d) 7,7 
15. Find two incomparable elements in these posets. 
a) (P({0, 1, 2}), ©) b) ({1, 2,4, 6, 8}, |) 


16. Let S = {1, 2, 3, 4}. With respect to the lexicographic or- 
der based on the usual less than elation, 
a) find all pairs in S x S less than (2, 3). 
b) find all pairs in S x S greater than (3, 1). 
c) draw the Hasse diagram of the poset (S x S$, =<). 

17. Find the lexicographic ordering of these n-tuples: 
a) (1, 1,2), (1, 2, 1) b) (0, 1, 2, 3), (0, 1, 3, 2) 
c) (1,0, 1,0, 1), (0, 1, 1, 1, 0) 

18. Find the lexicographic ordering of these strings of low- 
ercase English letters: 
a) quack, quick, quicksilver, quicksand, quacking 
b) open, opener, opera, operand, opened 
c) zoo, zero, zoom, zoology, zoological 

19. Find the lexicographic ordering of the bit strings 0, 01, 
11, 001, 010, 011, 0001, and 0101 based on the ordering 
0<1. 


20. Draw the Hasse diagram for the greater than or equal to 
relation on {0, 1, 2, 3, 4, 5}. 


21. Draw the Hasse diagram for the less than or equal to re- 
lation on {0, 2, 5, 10, 11, 15}. 


22. Draw the Hasse diagram for divisibility on the set 
a) {1,2,3,4,5, 6}. b) {3,5, 7, 11, 13, 16, 17}. 
c) {2,3,5, 10,11, 15,25}. d) {1,3,9, 27, 81, 243}. 
23. Draw the Hasse diagram for divisibility on the set 
a) {1,2,3,4,5,6,7, 8}. b) {1,2,3,5, 7, 11, 13}. 
c) {1,2,3, 6, 12, 24, 36, 48}. 
d) {1, 2, 4, 8, 16, 32, 64}. 
24. Draw the Hasse diagram for inclusion on the set P(S), 
where S = {a, b, c, d}. 
In Exercises 25-27 list all ordered pairs in the partial ordering 
with the accompanying Hasse diagram. 


25. 26. 
3 d e d 


27. 


a 
a 
<“ 


b a c 


28. What is the covering relation of the partial ordering 
{(a, b) | a divides b} on {1, 2, 3, 4, 6, 12}? 

29. What is the covering relation of the partial order- 
ing {(A, B)|A CB} on the power set of S, where 
S= {a,b,c}? 

30. What is the covering relation of the partial ordering for 
the poset of security classes defined in Example 25? 

31. Show that a finite poset can be reconstructed from its cov- 
ering relation. [Hint: Show that the poset is the reflexive 
transitive closure of its covering relation. ] 

32. Answer these questions for the partial order represented 
by this Hasse diagram. 


m 


a b ĉ 


a) Find the maximal elements. 

b) Find the minimal elements. 

c) Is there a greatest element? 

d) Is there a least element? 

e) Find all upper bounds of {a, b, c}. 

f) Find the least upper bound of {a, b, c}, if it exists. 
g) Find all lower bounds of {f g, h}. 

h) Find the greatest lower bound of {f, g, h}, if it exists. 
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33. Answer these questions for the poset ({3, 5,9,15, 
24, 45}, |). 
a) Find the maximal elements. 
b) Find the minimal elements. 
c) Is there a greatest element? 
d) Is there a least element? 
e) Find all upper bounds of {3, 5}. 
f) Find the least upper bound of {3, 5}, if it exists. 
g) Find all lower bounds of {15, 45}. 
h) Find the greatest lower bound of {15, 45}, if it exists. 
34. Answer these questions for the poset ({2, 4, 6,9, 12, 
18, 27, 36, 48, 60, 72}, |). 
a) Find the maximal elements. 
b) Find the minimal elements. 
c) Is there a greatest element? 
d) Is there a least element? 
e) Find all upper bounds of {2, 9}. 
f) Find the least upper bound of {2, 9}, if it exists. 
g) Find all lower bounds of {60, 72}. 
h) Find the greatest lower bound of {60, 72}, if it exists. 
35. Answer these questions for the poset ({{1}, {2}, {4}, 
{1,2}, {1,4}, {2,4}, {3,4}, (1,3, 4}, (2,3, 4}}, ©). 
a) Find the maximal elements. 
b) Find the minimal elements. 
c) Is there a greatest element? 
d) Is there a least element? 
e) Find all upper bounds of { {2}, {4}}. 
f) Find the least upper bound of { {2}, {4}}, if it exists. 
g) Find all lower bounds of {{1, 3, 4}, {2, 3, 4}}. 
h) Find the greatest lower bound of { {1, 3, 4}, {2, 3, 4}}, 
if it exists. 
36. Give a poset that has 
a) a minimal element but no maximal element. 
b) a maximal element but no minimal element. 
c) neither a maximal nor a minimal element. 
37. Show that lexicographic order is a partial ordering on the 
Cartesian product of two posets. 
38. Show that lexicographic order is a partial ordering on the 
set of strings from a poset. 
39. Suppose that (S, = ,) and (T, 3) are posets. Show that 
(S x T; x) is a poset where (s, t) = (u, v) if and only if 
s S,uandt <)v. 
40. a) Show that there is exactly one greatest element of a 
poset, if such an element exists. 
b) Show that there is exactly one least element of a poset, 
if such an element exists. 
41. a) Show that there is exactly one maximal element in a 
poset with a greatest element. 
b) Show that there is exactly one minimal element in a 
poset with a least element. 
42. a) Show that the least upper bound of a set in a poset is 
unique if it exists. 
b) Show that the greatest lower bound of a set in a poset 
is unique if it exists. 
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43. 


44. 


45. 


46. 


47. 


48. 


* 49, 


50. 
51. 


Determine whether the posets with these Hasse diagrams 
are lattices. 


a) b) c) 
g h i 
f f 8 h 
d . p : f g 
b é b FE A ° 
(6 


a 
a 


Determine whether these posets are lattices. 


a) ({1,3,6,9, 12}, |) b) ({1, 5,25, 125}, |) 
c) (Z, >) 
d) (P(S), 2), where P(S) is the power set of a set $ 


Show that every nonempty finite subset of a lattice has a 
least upper bound and a greatest lower bound. 


Show that if the poset (S, R) is a lattice then the dual poset 
(S, R7!) is also a lattice. 


In a company, the lattice model of information flow 
is used to control sensitive information with security 
classes represented by ordered pairs (A, C). Here A is an 
authority level, which may be nonproprietary (0), propri- 
etary (1), restricted (2), or registered (3). A category C 
is a subset of the set of all projects {Cheetah, Impala, 
Puma}. (Names of animals are often used as code names 
for projects in companies.) 

a) Is information permitted to flow from (Proprietary, 

{Cheetah, Puma}) into (Restricted, {Puma })? 


b) Is information permitted to flow from (Restricted, 
{ Cheetah}) into (Registered, { Cheetah, Impala})? 


c) Into which classes is information from (Proprietary, 
{ Cheetah, Puma}) permitted to flow? 


d) From which classes is information permitted to flow 
into the security class (Restricted, {Impala, Puma})? 


Show that the set S of security classes (A, C) is a lat- 
tice, where A is a positive integer representing an au- 
thority class and C is a subset of a finite set of com- 
partments, with (A,, C,) = (A, C,) ifand only ifA, < A, 
and C, C C). [Hint: First show that (S, = ) is a poset and 
then show that the least upper bound and greatest lower 
bound of (A,, C,) and (A), C,) are (max(A,, A,), Ci U C3) 
and (min(A,, A2), C; N C3), respectively.] 


Show that the set of all partitions of a set S with the re- 
lation P, = P, if the partition P} is a refinement of the 
partition P, is a lattice. (See the preamble to Exercise 49 
of Section 9.5.) 


Show that every totally ordered set is a lattice. 


Show that every finite lattice has a least element and a 
greatest element. 


52. 


53. 


54. 


Give an example of an infinite lattice with 
a) neither a least nor a greatest element. 


b) a least but not a greatest element. 
c) a greatest but not a least element. 


d) both a least and a greatest element. 


Verify that (Z*x Z*, =<) is a well-ordered set, where = 
is lexicographic order, as claimed in Example 8. 


Determine whether each of these posets is well-ordered. 
a) (S, <), where S = {10, 11, 12, ...} 


b) (Q N^ (0, 1], <) (the set of rational numbers between 0 
and 1 inclusive) 


c) (S, <), where S is the set of positive rational numbers 
with denominators not exceeding 3 


d) (Z7, >), where Z7 is the set of negative integers 


A poset (R, =<) is well-founded if there is no infinite de- 
creasing sequence of elements in the poset, that is, ele- 


ments x], Xz, . 


..,X, Such that «+» < x, < +++ < xX < x. A poset 


(R, =<) is dense if for all x € S and y E€ S with x < y, there is 
an element z € R such that x < z < y. 


55. 


56. 


57. 


* 58. 


59. 


60. 


6l. 


62. 


63. 


64. 


65. 


Show that the poset (Z, x), where x < y if and only if 
|x| < |y| is well-founded but is not a totally ordered set. 


Show that a dense poset with at least two elements that 
are comparable is not well-founded. 


Show that the poset of rational numbers with the usual 
less than or equal to relation, (Q, <), is a dense poset. 


Show that the set of strings of lowercase English let- 
ters with lexicographic order is neither well-founded nor 
dense. 


Show that a poset is well-ordered if and only if it is totally 
ordered and well-founded. 


Show that a finite nonempty poset has a maximal ele- 
ment. 


Find a compatible total order for the poset with the Hasse 
diagram shown in Exercise 32. 


Find a compatible total order for the divisibility relation 
on the set {1, 2, 3, 6, 8, 12, 24, 36}. 


Find all compatible total orderings for the poset 
({1, 2, 4, 5, 12, 20}, |} from Example 26. 


Find all compatible total orderings for the poset with the 
Hasse diagram in Exercise 27. 


Find all possible orders for completing the tasks in the 
development project in Example 27. 


66. Schedule the tasks needed to build a house, by specifying 
their order, if the Hasse diagram representing these tasks 
is as shown in the figure. 


Completion 


Interior fixtures 
Exterior fixtures 


Carpeting Interior painting 


Flooring Exterior painting 


Wall-board 


Plumbing Wiring 


Exterior siding 
Roof 
Framing 


Foundation 


Key Terms and Results 


TERMS 


binary relation from A to B: a subset of A x B 


relation on A: a binary relation from A to itself (that is, a sub- 
set of A x A) 


S oR: composite of R and S$ 
R-!: inverse relation of R 
R”: nth power of R 


reflexive: a relation R on A is reflexive if (a, a) E R for all 
acA 


symmetric: a relation R on A is symmetric if (b, a) € R when- 
ever (a, b) E R 


antisymmetric: a relation R on A is antisymmetric if a = b 
whenever (a, b) E R and (b, a) E R 

transitive: a relation R on A is transitive if (a, b) € R and 
(b, c) € R implies that (a,c) ER 

n-ary relation on A ,, A, ..., A„: a subset of A,X A, X + XA, 

relational data model: a model for representing databases us- 
ing n-ary relations 

primary key: a domain of an n-ary relation such that an n- 
tuple is uniquely determined by its value for this domain 


composite key: the Cartesian product of domains of an n-ary 
relation such that an n-tuple is uniquely determined by its 
values in these domains 

selection operator: a function that selects the n-tuples in an 
n-ary relation that satisfy a specified condition 

projection: a function that produces relations of smaller de- 
gree from an n-ary relation by deleting fields 

join: a function that combines n-ary relations that agree on 
certain fields 


itemset: a collection of items 
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67. Find an ordering of the tasks of a software project if the 
Hasse diagram for the tasks of the project is as shown. 


Completion 


B test 


atest 


Develop module A Integrate modules 


N 
Write Develop module B 
documentation Develop module C 


Set up 


Develop system I 
test sites 


requirements 
Write functional requirements 


Determine user needs 


count of an itemset: the number of transactions that are su- 
persets of the itemset 


frequent itemset: an itemset with frequency greater than or 
equal to the support threshold 


support of an itemset: the frequency of transactions that con- 
tain the itemset 


association rule: an implication of the form J > J, where I 
and J are itemsets 


support of the association rule J > J: the fraction of trans- 
actions that contain both the itemsets / and J 


confidence of an association rule: the conditional probability 
that J is a subset of a transaction given that J is 


directed graph or digraph: a set of elements called vertices 
and ordered pairs of these elements, called edges 


loop: an edge of the form (a, a) 


closure of a relation R with respect to a property P: the re- 
lation S (if it exists) that contains R, has property P, and 
is contained within any relation that contains R and has 
property P 

path in a digraph: a sequence of edges (a, x,), (%), X2), ..., 
Xn- X,_1)» (X,_1, b) such that the terminal vertex of each 
edge is the initial vertex of the succeeding edge in the se- 
quence 


circuit (or cycle) in a digraph: a path that begins and ends at 
the same vertex 

R* (connectivity relation): the relation consisting of those or- 
dered pairs (a, b) such that there is a path from a to b 

equivalence relation: a reflexive, symmetric, and transitive 
relation 


equivalent: if R is an equivalence relation, a is equivalent to 
bif aRb 
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[a] (equivalence class of a with respect to R): the set of all 
elements of A that are equivalent to a 


[a], (congruence class modulo m): the set of integers con- 
gruent to a modulo m 

partition of a set S: acollection of pairwise disjoint nonempty 
subsets that have S as their union 

partial ordering: a relation that is reflexive, antisymmetric, 
and transitive 

poset (S, R): a set S and a partial ordering R on this set 

comparable: the elements a and b in the poset (A, =) are 
comparable if a x borb x a 

incomparable: elements in a poset that are not comparable 

total (or linear) ordering: a partial ordering for which every 
pair of elements are comparable 

totally (or linearly) ordered set: a poset with a total (or lin- 
ear) ordering 

well-ordered set: a poset (S, =<), where = is a total order and 
every nonempty subset of S has a least element 

lexicographic order: a partial ordering of Cartesian products 
or strings 


Hasse diagram: a graphical representation of a poset where 
loops and all edges resulting from the transitive property 
are not shown, and the direction of the edges is indicated 
by the position of the vertices 

maximal element: an element of a poset that is not less than 
any other element of the poset 

minimal element: an element of a poset that is not greater than 
any other element of the poset 

greatest element: an element of a poset greater than all other 
elements in this set 


least element: an element of a poset less than all other ele- 
ments in this set 


Review Questions 


1. a) What is a relation on a set? 
b) How many relations are there on a set with n elements? 


2. a) What is a reflexive relation? 
b) What is a symmetric relation? 
c) What is an antisymmetric relation? 
d) What is a transitive relation? 


3. Give an example of a relation on the set {1, 2, 3, 4} that is 
a) reflexive, symmetric, and not transitive. 
b) not reflexive, symmetric, and transitive. 
c) reflexive, antisymmetric, and not transitive. 
d) reflexive, symmetric, and transitive. 
e) reflexive, antisymmetric, and transitive. 
4. a) How many reflexive relations are there on a set with n 
elements? 
b) How many symmetric relations are there on a set with 
n elements? 


upper bound of a set: an element in a poset greater than all 
other elements in the set 

lower bound of a set: an element in a poset less than all other 
elements in the set 

least upper bound of a set: an upper bound of the set that is 
less than all other upper bounds 

greatest lower bound of a set: a lower bound of the set that 
is greater than all other lower bounds 

lattice: a partially ordered set in which every two elements 
have a greatest lower bound and a least upper bound 

compatible total ordering for a partial ordering: a total or- 
dering that contains the given partial ordering 

topological sort: the construction of a total ordering compat- 
ible with a given partial ordering 


RESULTS 


The reflexive closure of a relation R on the set A equals R U A, 
where A = {(a,a) | a E A}. 

The symmetric closure of a relation R on the set A equals 
RUR™!, where R7! = {(b, a) | (a, b) € R}. 

The transitive closure of a relation equals the connectivity re- 
lation formed from this relation. 

Warshall’s algorithm for finding the transitive closure of a re- 
lation 

Let R be an equivalence relation. Then the following three 
statements are equivalent: (1) a R b; (2) [aly N [b] # ®; 
(3) [alr = [b]r. 

The equivalence classes of an equivalence relation on a set A 
form a partition of A. Conversely, an equivalence relation 
can be constructed from any partition so that the equiva- 
lence classes are the subsets in the partition. 

The principle of well-ordered induction 

The topological sorting algorithm 


c) How many antisymmetric relations are there on a set 
with n elements? 


5. a) Explain how an n-ary relation can be used to represent 
information about students at a university. 


b) How can the 5-ary relation containing names of stu- 
dents, their addresses, telephone numbers, majors, and 
grade point averages be used to form a 3-ary relation 
containing the names of students, their majors, and 
their grade point averages? 


c) How can the 4-ary relation containing names of stu- 
dents, their addresses, telephone numbers, and majors 
and the 4-ary relation containing names of students, 
their student numbers, majors, and numbers of credit 
hours be combined into a single n-ary relation? 


6. a) Explain how to use a zero—one matrix to represent a 
relation on a finite set. 


b) Explain how to use the zero—one matrix representing a 
relation to determine whether the relation is reflexive, 
symmetric, and/or antisymmetric. 


7. a) Explain how to use a directed graph to represent a re- 
lation on a finite set. 
b) Explain how to use the directed graph representing a 
relation to determine whether a relation is reflexive, 
symmetric, and/or antisymmetric. 


8. a) Define the reflexive closure and the symmetric closure 
of a relation. 

b) How can you construct the reflexive closure of a rela- 
tion? 

c) How can you construct the symmetric closure of a re- 
lation? 

d) Find the reflexive closure and the symmetric closure 
of the relation {(1, 2), (2, 3), (2, 4), (3, 1)} on the set 
{1, 2, 3, 4}. 

9. a) Define the transitive closure of a relation. 

b) Can the transitive closure of a relation be obtained 
by including all pairs (a, c) such that (a, b) and (b, c) 
belong to the relation? 

c) Describe two algorithms for finding the transitive 
closure of a relation. 

d) Find the transitive closure of the relation 
{(1,), (1,3), (2,1), (2,3), (2,4), 3,2), 3.4), 4,D}. 

10. a) Define an equivalence relation. 
b) Which relations on the set {a, b, c, d} are equivalence 

relations and contain (a, b) and (b, d)? 


11. a) Show that congruence modulo m is an equivalence re- 
lation whenever m is a positive integer. 

b) Show that the relation {(a, b) | a = +b (mod 7)} is an 
equivalence relation on the set of integers. 


12. a) What are the equivalence classes of an equivalence re- 
lation? 


Supplementary Exercises 


1. Let S be the set of all strings of English letters. Determine 
whether these relations are reflexive, irreflexive, symmet- 
ric, antisymmetric, and/or transitive. 

a) R, = {(a, b) | a and b have no letters in common} 
b) R, = {(a, b) | a and b are not the same length} 
c) R, = {(a, b) | ais longer than b} 

2. Construct a relation on the set {a, b, c, d} that is 

a) reflexive, symmetric, but not transitive. 

b) irreflexive, symmetric, and transitive. 

c) irreflexive, antisymmetric, and not transitive. 

d) reflexive, neither symmetric nor antisymmetric, and 
transitive. 

e) neither reflexive, irreflexive, symmetric, antisymmet- 
ric, nor transitive. 

3. Show that the relation R on Z x Z defined by (a, b) R (c, d) 
if and only if a + d = b + c is an equivalence relation. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 
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b) What are the equivalence classes of the “congruent 
modulo 5” relation? 

c) What are the equivalence classes of the equivalence 
relation in Question 11(b)? 

Explain the relationship between equivalence relations on 

a set and partitions of this set. 

a) Define a partial ordering. 

b) Show that the divisibility relation on the set of positive 
integers is a partial order. 

Explain how partial orderings on the sets A, and A, can 

be used to define a partial ordering on the set A, X Ap. 

a) Explain how to construct the Hasse diagram of a par- 
tial order on a finite set. 

b) Draw the Hasse diagram of the divisibility relation on 
the set {2, 3, 5, 9, 12, 15, 18}. 

a) Define a maximal element of a poset and the greatest 
element of a poset. 

b) Give an example of a poset that has three maximal el- 
ements. 

c) Give an example of a poset with a greatest element. 

a) Define a lattice. 

b) Give an example of a poset with five elements that is 
a lattice and an example of a poset with five elements 
that is not a lattice. 

a) Show that every finite subset of a lattice has a greatest 
lower bound and a least upper bound. 

b) Show that every lattice with a finite number of ele- 
ments has a least element and a greatest element. 

a) Define a well-ordered set. 

b) Describe an algorithm for producing a totally ordered 
set compatible with a given partially ordered set. 

c) Explain how the algorithm from (b) can be used to or- 
der the tasks in a project if tasks are done one at a time 
and each task can be done only after one or more of 
the other tasks have been completed. 


. Show that a subset of an antisymmetric relation is also 


antisymmetric. 


5. Let R bea reflexive relation on a set A. Show that R C R?. 


10. 


. Suppose that R, and R, are reflexive relations on a set A. 


Show that R; ® R, is irreflexive. 


. Suppose that R, and R, are reflexive relations on a set A. 


Is R| N R, also reflexive? Is R} U R, also reflexive? 


. Suppose that R is a symmetric relation on a set A. Is R 


also symmetric? 


. Let R, and R, be symmetric relations. Is R; N R, also 


symmetric? Is R; U R, also symmetric? 


A relation R is called circular if aRb and bRc imply that 
cRa. Show that R is reflexive and circular if and only if it 
is an equivalence relation. 
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11. 


12. 


13. 


14. 


15. 


Show that a primary key in an n-ary relation is a primary 16. a) Let S be the set of subroutines of a computer program. 
key in any projection of this relation that contains this key Define the relation R by P RQ if subroutine P calls 
as one of its fields. subroutine Q during its execution. Describe the tran- 


sitive closure of R. 


b) For which subroutines P does (P, P) belong to the 
transitive closure of R? 


Is the primary key in an n-ary relation also a primary key 
in a larger relation obtained by taking the join of this re- 
lation with a second relation? 


c) Describe the reflexive closure of the transitive closure 
Show that the reflexive closure of the symmetric closure of R. 


of a relation is the same as the symmetric closure of its 


reflexive closure. 17. Suppose that R and S are relations on a set A with R C S 


such that the closures of R and S with respect to a prop- 


Let R be the relation on the set of all mathematicians that erty P both exist. Show that the closure of R with respect 
contains the ordered pair (a, b) if and only if a and b have to P is a subset of the closure of S with respect to P. 
written a published mathematical paper together, 18. Show that the symmetric closure of the union of two re- 


a) Describe the relation R^. lations is the union of their symmetric closures. 


b) Describe the relation R*. *19, Devise an algorithm, based on the concept of interior ver- 
tices, that finds the length of the longest path between two 
vertices in a directed graph, or determines that there are 
arbitrarily long paths between these vertices. 


c) The Erdős number of a mathematician is 1 if this 
mathematician wrote a paper with the prolific Hun- 
garian mathematician Paul Erdős, it is 2 if this math- 
ematician did not write a joint paper with Erdés but 20. Which of these are equivalence relations on the set of all 
wrote a joint paper with someone who wrote a joint people? 
paper with Erdős, and so on (except that the Erdős 
number of Erdős himself is 0). Give a definition of 
the Erdős number in terms of paths in R. 


a) {(x, y) | x and y have the same sign of the zodiac } 


b) {(x, y) | x and y were born in the same year} 


a) Give an example to show that the transitive closure c) {(, y) | x and y have been in the same city} 
of the symmetric closure of a relation is not necessar- *21. How many different equivalence relations with exactly 
ily the same as the symmetric closure of the transitive three different equivalence classes are there on a set with 
closure of this relation. five elements? 

b) Show, however, that the transitive closure of the sym- 22. Show that {(x, y) | x — y E€ Q} is an equivalence relation 
metric closure of a relation must contain the symmet- on the set of real numbers, where Q denotes the set of 
ric closure of the transitive closure of this relation. rational numbers. What are [1], 5], and [z]? 


Links » 


PAUL ERDŐS (1913-1996) Paul Erdős, born in Budapest, Hungary, was the son of two high school mathe- 
matics teachers. He was a child prodigy; at age 3 he could multiply three-digit numbers in his head, and at 4 he 
discovered negative numbers on his own. Because his mother did not want to expose him to contagious diseases, 
he was mostly home-schooled. At 17 Erdős entered Eőtvős University, graduating four years later with a Ph.D. 
in mathematics. After graduating he spent four years at Manchester, England, on a postdoctoral fellowship. In 
1938 he went to the United States because of the difficult political situation in Hungary, especially for Jews. 
He spent much of his time in the United States, except for 1954 to 1962, when he was banned as part of the 
paranoia of the McCarthy era. He also spent considerable time in Israel. 

Erdős made many significant contributions to combinatorics and to number theory. One of the discoveries 
of which he was most proud is his elementary proof (in the sense that it does not use any complex analysis) 
of the prime number theorem, which provides an estimate for the number of primes not exceeding a fixed positive integer. He also 
participated in the modern development of the Ramsey theory. 

Erdés traveled extensively throughout the world to work with other mathematicians, visiting conferences, universities, and 
research laboratories. He had no permanent home. He devoted himself almost entirely to mathematics, traveling from one mathe- 
matician to the next, proclaiming “My brain is open.” Erdős was the author or coauthor of more than 1500 papers and had more 
than 500 coauthors. Copies of his articles are kept by Ron Graham, a famous discrete mathematician with whom he collaborated 
extensively and who took care of many of his worldly needs. 

Erdős offered rewards, ranging from $10 to $10,000, for the solution of problems that he found particularly interesting, with the 
size of the reward depending on the difficulty of the problem. He paid out close to $4000. Erdős had his own special language, using 
such terms as “epsilon” (child), “boss” (woman), “slave” (man), “captured” (married), “liberated” (divorced), “Supreme Fascist” 
(God), “Sam” (United States), and “Joe” (Soviet Union). Although he was curious about many things, he concentrated almost all 
his energy on mathematical research. He had no hobbies and no full-time job. He never married and apparently remained celibate. 
Erdős was extremely generous, donating much of the money he collected from prizes, awards, and stipends for scholarships and to 
worthwhile causes. He traveled extremely lightly and did not like having many material possessions. 


Courtesy of George Csicsery 


23. Suppose that P,={Aj,,Aj,...,A,,} and Py, = 
{B,, Ba, ..., B,} are both partitions of the set S. Show that 
the collection of nonempty subsets of the form A; N B; is 
a partition of S that is a refinement of both P, and P, (see 


the preamble to Exercise 49 of Section 9.5). 


*24, Show that the transitive closure of the symmetric clo- 
sure of the reflexive closure of a relation R is the smallest 
equivalence relation that contains R. 


25. Let R(S) be the set of all relations on a set S. Define the 
relation < on R(S) by R; = Ry if R; C Ry, where R} and 
R, are relations on S. Show that (R(S), = ) is a poset. 


26. Let P(S) be the set of all partitions of the set S. Define the 
relation = on P(S) by P} = P; if P, is a refinement of 
P, (see Exercise 49 of Section 9.5). Show that (P(S), x) 
is a poset. 


27. Schedule the tasks needed to cook a Chinese meal by 
specifying their order, if the Hasse diagram representing 


these tasks is as shown here. 
Serve 


Arrange on platters 


Cook in wok 


Cut 
fish 


Clean 
fish 


Cut ginger 
and garlic 


Buy groceries 


Buy seafood 


Find recipe 


A subset of a poset such that every two elements of this sub- 
set are comparable is called a chain. A subset of a poset is 
called an antichain if every two elements of this subset are 
incomparable. 


28. Find all chains in the posets with the Hasse diagrams 
shown in Exercises 25—27 in Section 9.6. 


29. Find all antichains in the posets with the Hasse diagrams 
shown in Exercises 25—27 in Section 9.6. 


30. Find an antichain with the greatest number of elements 
in the poset with the Hasse diagram of Exercise 32 in 
Section 9.6. 


31. Show that every maximal chain in a finite poset (S, = ) 
contains a minimal element of S. (A maximal chain is a 
chain that is not a subset of a larger chain.) 
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** 32. Show that every finite poset can be partitioned into k 


chains, where k is the largest number of elements in an 
antichain in this poset. 


* 33. Show that in any group of mn + 1 people there is either a 
list of m + 1 people where a person in the list (except for 
the first person listed) is a descendant of the previous per- 
son on the list, or there are n + 1 people such that none of 
these people is a descendant of any of the other people. 
[Hint: Use Exercise 32.] 

Suppose that (S,  ) is a well-founded partially ordered set. 

The principle of well-founded induction states that P(x) is true 

for all x E€ S if Vx(Vy(y < x > P(y)) > Px). 

34. Show that no separate basis case is needed for the princi- 
ple of well-founded induction. That is, P(u) is true for all 
minimal elements u in S if Vx(Vy(y < x > P(y)) > P(x)). 


*35. Show that the principle of well-founded induction is 
valid. 


A relation R on a set A is a quasi-ordering on A if R is reflex- 

ive and transitive. 

36. Let R be the relation on the set of all functions from 
Z* to Z* such that (f, g) belongs to R if and only if f 
is O(g). Show that R is a quasi-ordering. 

37. Let R be a quasi-ordering on a set A. Show that R n R7! 
is an equivalence relation. 

* 38. Let R be a quasi-ordering and let S be the relation on 
the set of equivalence classes of R N R7! such that (C, D) 
belongs to S, where C and D are equivalence classes 
of R, if and only if there are elements c of C and d of 
D such that (c, d) belongs to R. Show that S is a partial 
ordering. 

Let L be a lattice. Define the meet (A) and join (V) operations 

by x A y = glb(@, y) and x V y = lub(x, y). 

39. Show that the following properties hold for all elements 
x, y, and z of a lattice L. 

a) xAy=yAxandxVy=yV x (commutative laws) 

b) WA Y)AZ=XA(WAZD and WV y)VZ=XVU—VV2Z) 
(associative laws) 

c) xA(xVy) = xand xV (x A y) = x (absorption laws) 

d) x^ x = x and x V x = x (idempotent laws) 

40. Show that if x and y are elements of a lattice L, then 
xV y = y if and only if x A y = x. 

A lattice L is bounded if it has both an upper bound, de- 

noted by 1, such that x =< 1 for all x € L and a lower bound, 

denoted by 0, such that 0 x x for all x € L. 

41. Show that if L is a bounded lattice with upper bound 
1 and lower bound 0 then these properties hold for all 
elements x E L. 

a) xvl=l1 b) xAlL=x 
c) xVO=x d) xA0=0 

42. Show that every finite lattice is bounded. 

A lattice is called distributive if x V (y A z) = (x Vy)A(xV 2) 

and x A (y Vz) = (x A y) V (x A z) for all x, y, and z in L. 


*43. Give an example of a lattice that is not distributive. 
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44. Show that the lattice (P(S), C) where P(S) is the power 
set of a finite set S is distributive. 

45. Is the lattice (Z*, |) distributive? 

The complement of an element a of a bounded lattice L with 

upper bound 1 and lower bound 0 is an element b such that 

aVb=1andaAb=0. Such a lattice is complemented if 
every element of the lattice has a complement. 

46. Give an example of a finite lattice where at least one el- 
ement has more than one complement and at least one 
element has no complement. 

47. Show that the lattice (P(S), C) where P(S) is the power 
set of a finite set S is complemented. 

* 48. Show that if L is a finite distributive lattice, then an ele- 
ment of L has at most one complement. 


Computer Projects 


Write programs with these input and output. 


1. Given the matrix representing a relation on a finite 
set, determine whether the relation is reflexive and/or 
irreflexive. 


2. Given the matrix representing a relation on a finite set, 
determine whether the relation is symmetric and/or anti- 
symmetric. 


3. Given the matrix representing a relation on a finite set, 
determine whether the relation is transitive. 


4. Given a positive integer n, display all the relations on a 
set with n elements. 


*5. Given a positive integer n, determine the number of tran- 
sitive relations on a set with n elements. 


*6. Given a positive integer n, determine the number of 
equivalence relations on a set with n elements. 


*7, Given a positive integer n, display all the equivalence re- 
lations on the set of the smallest positive integers. 


8. Given an n-ary relation, find the projection of this relation 
when specified fields are deleted. 


Computations and Explorations 


The game of Chomp, introduced in Example 12 in Section 
1.8, can be generalized for play on any finite partially ordered 
set (S, <) with a least element a. In this game, a move consists 
of selecting an element x in S and removing x and all elements 
larger than it from S. The loser is the player who is forced to 
select the least element a. 


49. Show that the game of Chomp with cookies arranged in 
an m Xn rectangular grid, described in Example 12 in 
Section 1.8, is the same as the game of Chomp on the 
poset (S, |), where S is the set of all positive integers that 
divide p”—'q""', where p and q are distinct primes. 


50. Show that if (S, <) has a greatest element b, then a win- 
ning strategy for Chomp on this poset exists. [Hint: Gen- 
eralize the argument in Example 12 in Section 1.8.] 


9. Given an m-ary relation and an n-ary relation, and a set 
of common fields, find the join of these relations with re- 
spect to these common fields. 

10. Given the matrix representing a relation on a finite set, 
find the matrix representing the reflexive closure of this 
relation. 


11. Given the matrix representing a relation on a finite set, 
find the matrix representing the symmetric closure of this 
relation. 


12. Given the matrix representing a relation on a finite set, 
find the matrix representing the transitive closure of this 
relation by computing the join of the Boolean powers of 
the matrix representing the relation. 

13. Given the matrix representing a relation on a finite set, 
find the matrix representing the transitive closure of this 
relation using Warshall’s algorithm. 

14. Given the matrix representing a relation on a finite set, 
find the matrix representing the smallest equivalence re- 
lation containing this relation. 

15. Given a partial ordering on a finite set, find a total order- 
ing compatible with it using topological sorting. 


Use a computational program or programs you have written to do these exercises. 


1. Display all the different relations on a set with four ele- 
ments. 


2. Display all the different reflexive and symmetric relations 
on a set with six elements. 


3. Display all the reflexive and transitive relations on a set 
with five elements. 


*4, Determine how many transitive relations there are on a set 


with n elements for all positive integers n with n < 7. 


5. Find the transitive closure of a relation of your choice on 
a set with at least 20 elements. Either use a relation that 
corresponds to direct links in a particular transportation or 
communications network or use a randomly generated re- 
lation. 


6. Compute the number of different equivalence relations 
on a set with n elements for all positive integers n not 
exceeding 20. 

7. Display all the equivalence relations on a set with seven 
elements. 


Writing Projects 
Respond to these with essays using outside sources. 


1. Discuss the concept of a fuzzy relation. How are fuzzy 
relations used? 


2. Describe the basic principles of relational databases, go- 
ing beyond what was covered in Section 9.2. How widely 
used are relational databases as compared with other 
types of databases? 


3. Explain how the Apriori algorithm is used to find fre- 
quent itemsets and strong association rules. 


4. Describe some applications of association rules in detail. 


5. Look up the original papers by Warshall and by Roy 
(in French) in which they develop algorithms for finding 
transitive closures. Discuss their approaches. Why do you 
suppose that what we call Warshall’s algorithm was dis- 
covered independently by more than one person? 


6. Describe how equivalence classes can be used to define 
the rational numbers as classes of pairs of integers and 
how the basic arithmetic operations on rational numbers 
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*8. Display all the partial orders on a set with five elements. 


*9, Display all the lattices on a set with five elements. 


10. 


11. 


12. 


can be defined following this approach. (See Exercise 40 
in Section 9.5.) 


. Explain how Helmut Hasse used what we now call Hasse 


diagrams. 


. Describe some of the mechanisms used to enforce infor- 


mation flow policies in computer operating systems. 


. Discuss the use of the Program Evaluation and Review 


Technique (PERT) to schedule the tasks of a large com- 
plicated project. How widely is PERT used? 

Discuss the use of the Critical Path Method (CPM) to find 
the shortest time for the completion of a project. How 
widely is CPM used? 

Discuss the concept of duality in a lattice. Explain how 
duality can be used to establish new results. 

Explain what is meant by a modular lattice. Describe 
some of the properties of modular lattices and describe 
how modular lattices arise in the study of projective 
geometry. 


10.1 Graphs and 
Graph Models 


10.2 Graph Termi- 
nology and 
Special Types 
of Graphs 


10.3 Representing 
Graphs and 
Graph Iso- 
morphism 


10.4 Connectivity 


10.5 Euler and 
Hamilton 
Paths 


10.6 Shortest-Path 
Problems 


10.7 Planar Graphs 


10.8 Graph Color- 
ing 


Graphs 


{ raphs are discrete structures consisting of vertices and edges that connect these vertices. 

There are different kinds of graphs, depending on whether edges have directions, whether 
multiple edges can connect the same pair of vertices, and whether loops are allowed. Problems 
in almost every conceivable discipline can be solved using graph models. We will give examples 
to illustrate how graphs are used as models in a variety of areas. For instance, we will show how 
graphs are used to represent the competition of different species in an ecological niche, how 
graphs are used to represent who influences whom in an organization, and how graphs are used 
to represent the outcomes of round-robin tournaments. We will describe how graphs can be 
used to model acquaintanceships between people, collaboration between researchers, telephone 
calls between telephone numbers, and links between websites. We will show how graphs can 
be used to model roadmaps and the assignment of jobs to employees of an organization. 

Using graph models, we can determine whether it is possible to walk down all the streets 
in a city without going down a street twice, and we can find the number of colors needed to 
color the regions of a map. Graphs can be used to determine whether a circuit can be imple- 
mented on a planar circuit board. We can distinguish between two chemical compounds with the 
same molecular formula but different structures using graphs. We can determine whether two 
computers are connected by a communications link using graph models of computer networks. 
Graphs with weights assigned to their edges can be used to solve problems such as finding the 
shortest path between two cities in a transportation network. We can also use graphs to schedule 
exams and assign channels to television stations. This chapter will introduce the basic concepts 
of graph theory and present many different graph models. To solve the wide variety of problems 
that can be studied using graphs, we will introduce many different graph algorithms. We will 
also study the complexity of these algorithms. 


Graphs and Graph Models 


Definition 1 


We begin with the definition of a graph. 


A graph G = (V, E) consists of V, a nonempty set of vertices (or nodes) and E, a set of edges. 
Each edge has either one or two vertices associated with it, called its endpoints. An edge is 
said to connect its endpoints. 


Remark: The set of vertices V of a graph G may be infinite. A graph with an infinite vertex 
set or an infinite number of edges is called an infinite graph, and in comparison, a graph with 
a finite vertex set and a finite edge set is called a finite graph. In this book we will usually 
consider only finite graphs. 


Now suppose that a network is made up of data centers and communication links between 
computers. We can represent the location of each data center by a point and each communica- 
tions link by a line segment, as shown in Figure 1. 

This computer network can be modeled using a graph in which the vertices of the graph 
represent the data centers and the edges represent communication links. In general, we visualize 
graphs by using points to represent vertices and line segments, possibly curved, to represent 
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Detroit 


New York 
San Francisco 


Washington 


Los Angeles 


FIGURE 1 A computer network. 


edges, where the endpoints of a line segment representing an edge are the points representing 
the endpoints of the edge. When we draw a graph, we generally try to draw edges so that they do 
not cross. However, this is not necessary because any depiction using points to represent vertices 
and any form of connection between vertices can be used. Indeed, there are some graphs that 
cannot be drawn in the plane without edges crossing (see Section 10.7). The key point is that 
the way we draw a graph is arbitrary, as long as the correct connections between vertices are 
depicted. 

Note that each edge of the graph representing this computer network connects two different 
vertices. That is, no edge connects a vertex to itself. Furthermore, no two different edges connect 
the same pair of vertices. A graph in which each edge connects two different vertices and where 
no two edges connect the same pair of vertices is called a simple graph. Note that in a simple 
graph, each edge is associated to an unordered pair of vertices, and no other edge is associated 
to this same edge. Consequently, when there is an edge of a simple graph associated to {u, v}, 
we can also say, without possible confusion, that {u, v} is an edge of the graph. 

A computer network may contain multiple links between data centers, as shown in Figure 2. 
To model such networks we need graphs that have more than one edge connecting the same 
pair of vertices. Graphs that may have multiple edges connecting the same vertices are called 
multigraphs. When there are m different edges associated to the same unordered pair of vertices 
{u, v}, we also say that {u, v} is an edge of multiplicity m. That is, we can think of this set of 
edges as m different copies of an edge {u, v}. 


Deron New York 


Chicago 


San Francisco 
Washington 


Los Angeles 
FIGURE2 A computer network with multiple links between data centers. 


Sometimes a communications link connects a data center with itself, perhaps a feedback 
loop for diagnostic purposes. Such a network is illustrated in Figure 3. To model this network we 
need to include edges that connect a vertex to itself. Such edges are called loops, and sometimes 
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FIGURE 3 A computer network with diagnostic links. 
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FIGURE 4 A communications network with one-way communications links. 


we may even have more than one loop at a vertex. Graphs that may include loops, and possibly 
multiple edges connecting the same pair of vertices or a vertex to itself, are sometimes called 
pseudographs. 

So far the graphs we have introduced are undirected graphs. Their edges are also said 
to be undirected. However, to construct a graph model, we may find it necessary to assign 
directions to the edges of a graph. For example, in a computer network, some links may operate 
in only one direction (such links are called single duplex lines). This may be the case if there is 
a large amount of traffic sent to some data centers, with little or no traffic going in the opposite 
direction. Such a network is shown in Figure 4. 

To model such a computer network we use a directed graph. Each edge of a directed graph 
is associated to an ordered pair. The definition of directed graph we give here is more general 
than the one we used in Chapter 9, where we used directed graphs to represent relations. 


A directed graph (or digraph) (V, E) consists of a nonempty set of vertices V and a set of 
directed edges (or arcs) E. Each directed edge is associated with an ordered pair of vertices. 
The directed edge associated with the ordered pair (u, v) is said to start at u and end at v. 


When we depict a directed graph with a line drawing, we use an arrow pointing from u to v to 
indicate the direction of an edge that starts at u and ends at v. A directed graph may contain loops 
and it may contain multiple directed edges that start and end at the same vertices. A directed 
graph may also contain directed edges that connect vertices u and v in both directions; that is, 
when a digraph contains an edge from u to v, it may also contain one or more edges from v to u. 
Note that we obtain a directed graph when we assign a direction to each edge in an undirected 
graph. When a directed graph has no loops and has no multiple directed edges, it is called a 
simple directed graph. Because a simple directed graph has at most one edge associated to each 
ordered pair of vertices (u, v), we call (u, v) an edge if there is an edge associated to it in the graph. 

In some computer networks, multiple communication links between two data centers may 
be present, as illustrated in Figure 5. Directed graphs that may have multiple directed edges 
from a vertex to a second (possibly the same) vertex are used to model such networks. We 
called such graphs directed multigraphs. When there are m directed edges, each associated to 
an ordered pair of vertices (u, v), we say that (u, v) is an edge of multiplicity m. 

For some models we may need a graph where some edges are undirected, while others are 
directed. A graph with both directed and undirected edges is called a mixed graph. For example, 
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FIGURE 5 A computer network with multiple one-way links. 
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TABLE 1 Graph Terminology. 


Simple graph Undirected 
Multigraph Undirected 
Pseudograph Undirected 
Simple directed graph Directed 


Directed multigraph Directed 


Mixed graph Directed and undirected 


a mixed graph might be used to model a computer network containing links that operate in both 
directions and other links that operate only in one direction. 
This terminology for the various types of graphs is summarized in Table 1. We will some- 

times use the term graph as a general term to describe graphs with directed or undirected edges 
(or both), with or without loops, and with or without multiple edges. At other times, when the 
context is clear, we will use the term graph to refer only to undirected graphs. 

Li » Because of the relatively modern interest in graph theory, and because it has applications 

inks : . paren . f : : 

to a wide variety of disciplines, many different terminologies of graph theory have been in- 
troduced. The reader should determine how such terms are being used whenever they are en- 
countered. The terminology used by mathematicians to describe graphs has been increasingly 
standardized, but the terminology used to discuss graphs when they are used in other disci- 
plines is still quite varied. Although the terminology used to describe graphs may vary, three 
key questions can help us understand the structure of a graph: 


> Are the edges of the graph undirected or directed (or both)? 


> If the graph is undirected, are multiple edges present that connect the same pair of ver- 
tices? If the graph is directed, are multiple directed edges present? 


> Are loops present? 


Answering such questions helps us understand graphs. It is less important to remember the 
particular terminology used. 


10.1.1 Graph Models 


TA » Graphs are used in a wide variety of models. We began this section by describing how to con- 
struct graph models of communications networks linking data centers. We will complete this 
section by describing some diverse graph models for some interesting applications. We will 
return to many of these applications later in this chapter and in Chapter 11. We will introduce 

l additional graph models in subsequent sections of this and later chapters. Also, recall that di- 
Can you find a subject ae : : : 
aNG rected graph models for some applications were introduced in Chapter 9. When we build a graph 
has not been applied? model, we need to make sure that we have correctly answered the three key questions we posed 
about the structure of a graph. 


SOCIAL NETWORKS Graphs are extensively used to model social structures based on dif- 
ferent kinds of relationships between people or groups of people. These social structures, and the 
graphs that represent them, are known as social networks. In these graph models, individuals 
or organizations are represented by vertices; relationships between individuals or organizations 
are represented by edges. The study of social networks is an extremely active multidisciplinary 
area, and many different types of relationships between people have been studied using them. 
We will introduce some of the most commonly studied social networks here. More information 
about social networks can be found in [Ne10] and [EaKĶ110]. 
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FIGURE 6 An acquaintanceship graph. FIGURE7 An influence graph. 
EXAMPLE 1 Acquaintanceship and Friendship Graphs We can use a simple graph to represent whether 


Links » 


EXAMPLE 2 


EXAMPLE 3 


Links >» 


two people know each other, that is, whether they are acquainted, or whether they are friends 
(either in the real world or in the virtual world via a social networking site such as Facebook). 
Each person in a particular group of people is represented by a vertex. An undirected edge is 
used to connect two people when these people know each other, when we are concerned only 
with acquaintanceship, or whether they are friends. No multiple edges and usually no loops are 
used. (If we want to include the notion of self-knowledge, we would include loops.) A small 
acquaintanceship graph is shown in Figure 6. The acquaintanceship graph of all people in the 
world has more than six billion vertices and probably more than one trillion edges! We will 
discuss this graph further in Section 10.4. < 


Influence Graphs In studies of group behavior it is observed that certain people can influence 
the thinking of others. A directed graph called an influence graph can be used to model this 
behavior. Each person of the group is represented by a vertex. There is a directed edge from 
vertex a to vertex b when the person represented by vertex a can influence the person represented 
by vertex b. This graph does not contain loops and it does not contain multiple directed edges. 
An example of an influence graph for members of a group is shown in Figure 7. In the group 
modeled by this influence graph, Deborah cannot be influenced, but she can influence Brian, 
Fred, and Linda. Also, Yvonne and Brian can influence each other. < 


Collaboration Graphs A collaboration graph is used to model social networks where two 
people are related by working together in a particular way. Collaboration graphs are simple 
graphs, as edges in these graphs are undirected and there are no multiple edges or loops. Vertices 
in these graphs represent people; two people are connected by an undirected edge when the 
people have collaborated. There are no loops nor multiple edges in these graphs. The Hollywood 
graph is a collaborator graph that represents actors by vertices and connects two actors with 
an edge if they have worked together on a movie or television show. The Hollywood graph is a 
huge graph with more than 2.9 million vertices (as of early 2018). We will discuss some aspects 
of the Hollywood graph later in Section 10.4. 

In an academic collaboration graph, vertices represent people (perhaps restricted to 
members of a certain academic community), and edges link two people if they have jointly 
published a paper. The collaboration graph for people who have published research papers in 
mathematics was found in 2004 to have more than 400,000 vertices and 675,000 edges, and 
these numbers have grown considerably since then. We will have more to say about this graph 
in Section 10.4. Collaboration graphs have also been used in sports, where two professional 
athletes are considered to have collaborated if they have ever played on the same team during 
a regular season of their sport. < 
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EXAMPLE 6 


732-555-1234 


732-555-9876 


COMMUNICATION NETWORKS We can model different communications networks using 
vertices to represent devices and edges to represent the particular type of communications links 
of interest. We have already modeled a data network in the first part of this section. 


Call Graphs Graphs can be used to model telephone calls made in a network, such as a long- 
distance telephone network. In particular, a directed multigraph can be used to model calls where 
each telephone number is represented by a vertex and each telephone call is represented by a 
directed edge. The edge representing a call starts at the telephone number from which the call 
was made and ends at the telephone number to which the call was made. We need directed edges 
because the direction in which the call is made matters. We need multiple directed edges because 
we want to represent each call made from a particular telephone number to a second number. 

A small telephone call graph is displayed in Figure 8(a), representing seven telephone 
numbers. This graph shows, for instance, that three calls have been made from 732-555-1234 
to 732-555-9876 and two in the other direction, but no calls have been made from 732-555- 
4444 to any of the other six numbers except 732-555-0011. When we care only whether there 
has been a call connecting two telephone numbers, we use an undirected graph with an edge 
connecting telephone numbers when there has been a call between these numbers. This version 
of the call graph is displayed in Figure 8(b). 

Call graphs that model actual calling activities can be huge. For example, one call graph 
studied at AT&T, which models calls during 20 days, has about 290 million vertices and 
4 billion edges. We will discuss call graphs further in Section 10.4. < 


INFORMATION NETWORKS Graphs can be used to model various networks that link par- 
ticular types of information. Here, we will describe how to model the web using a graph. We 
will also describe how to use a graph to model the citations in different types of documents. 


The Web Graph The web can be modeled as a directed graph where each web page is repre- 
sented by a vertex and where an edge starts at the web page a and ends at the web page b if there 
is a link on a pointing to b. Because new web pages are created and others removed somewhere 
on the web almost every second, the web graph changes on an almost continual basis. Many 
people are studying the properties of the web graph to better understand the nature of the web. 
We will return to web graphs in Section 10.4, and in Chapter 11 we will explain how the web 
graph is used by the web crawlers that search engines use to create indexes of web pages. <4 


Citation Graphs Graphs can be used to represent citations in different types of documents, 
including academic papers, patents, and legal opinions. In such graphs, each document is rep- 
resented by a vertex, and there is an edge from one document to a second document if the first 


732-555-1001 732-555-1001 
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FIGURE 8 A call graph. 
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document cites the second in its citation list. (In an academic paper, the citation list is the bib- 
liography, or list of references; in a patent it is the list of previous patents that are cited; and 
in a legal opinion it is the list of previous opinions cited.) A citation graph is a directed graph 
without loops or multiple edges. < 


SOFTWARE DESIGN APPLICATIONS Graph models are useful tools in the design of soft- 
ware. We will briefly describe two of these models here. 


Module Dependency Graphs One of the most important tasks in designing software is how 
to structure a program into different parts, or modules. Understanding how the different mod- 
ules of a program interact is essential not only for program design, but also for testing and 
maintenance of the resulting software. A module dependency graph provides a useful tool for 
understanding how different modules of a program interact. In a program dependency graph, 
each module is represented by a vertex. There is a directed edge from a module to a second 
module if the second module depends on the first. An example of a program dependency graph 
for a web browser is shown in Figure 9. 


Precedence Graphs and Concurrent Processing Computer programs can be executed more 
rapidly by executing certain statements concurrently. It is important not to execute a statement 
that requires results of statements not yet executed. The dependence of statements on previous 
statements can be represented by a directed graph. Each statement is represented by a vertex, 
and there is an edge from one statement to a second statement if the second statement cannot 
be executed before the first statement. This resulting graph is called a precedence graph. A 
computer program and its graph are displayed in Figure 10. For instance, the graph shows that 
statement S; cannot be executed before statements S,, S,, and S, are executed. < 


TRANSPORTATION NETWORKS We can use graphs to model many different types of 
transportation networks, including road, air, and rail networks, as well shipping networks. 


Airline Routes We can model airline networks by representing each airport by a vertex. 
In particular, we can model all the flights by a particular airline each day using a directed 
edge to represent each flight, going from the vertex representing the departure airport to the 
vertex representing the destination airport. The resulting graph will generally be a directed 
multigraph, as there may be multiple flights from one airport to some other airport during the 


same day. < 
So 
main 
Sı a:=0 Ss 
Sy b:=1 
display parser protocol S3 c:=atl 
S, d:=b+a OS 
Ss e:=d+l1 
Sg e:=ctd 
abstract page network 
syntax tree S| S5 


FIGURE 9 A module dependency graph. FIGURE 10 A precedence graph. 
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Road Networks Graphs can be used to model road networks. In such models, vertices repre- 
sent intersections and edges represent roads. When all roads are two-way and there is at most 
one road connecting two intersections, we can use a simple undirected graph to model the road 
network. However, we will often want to model road networks when some roads are one-way 
and when there may be more than one road between two intersections. To build such models, we 
use undirected edges to represent two-way roads and we use directed edges to represent one- 
way roads. Multiple undirected edges represent multiple two-way roads connecting the same 
two intersections. Multiple directed edges represent multiple one-way roads that start at one 
intersection and end at a second intersection. Loops represent loop roads. Mixed graphs are 
needed to model road networks that include both one-way and two-way roads. < 


BIOLOGICAL NETWORKS Many aspects of the biological sciences can be modeled using 
graphs. 


Niche Overlap Graphs in Ecology Graphs are used in many models involving the interaction 
of different species of animals. For instance, the competition between species in an ecosystem 
can be modeled using a niche overlap graph. Each species is represented by a vertex. An 
undirected edge connects two vertices if the two species represented by these vertices compete 
(that is, some of the food resources they use are the same). A niche overlap graph is a simple 
graph because no loops or multiple edges are needed in this model. The graph in Figure 11 
models the ecosystem of a forest. We see from this graph that squirrels and raccoons compete 
but that crows and shrews do not. < 


Protein Interaction Graphs A protein interaction in a living cell occurs when two or more 
proteins in that cell bind to perform a biological function. Because protein interactions are cru- 
cial for most biological functions, many scientists work on discovering new proteins and under- 
standing interactions between proteins. Protein interactions within a cell can be modeled using a 
protein interaction graph (also called a protein-protein interaction network), an undirected 
graph in which each protein is represented by a vertex, with an edge connecting the vertices rep- 
resenting each pair of proteins that interact. It is a challenging problem to determine genuine 
protein interactions in a cell, as experiments often produce false positives, which conclude that 
two proteins interact when they really do not. Protein interaction graphs can be used to deduce 
important biological information, such as by identifying the most important proteins for various 
functions and the functionality of newly discovered proteins. 

Because there are thousands of different proteins in a typical cell, the protein interaction 
graph of a cell is extremely large and complex. For example, yeast cells have more than 6,000 
proteins, and more than 80,000 interactions between them are known, and human cells have 
more than 100,000 proteins, with perhaps as many as 1,000,000 interactions between them. 
Additional vertices and edges are added to a protein interaction graph when new proteins and 
interactions between proteins are discovered. Because of the complexity of protein interac- 
tion graphs, they are often split into smaller graphs called modules that represent groups of 
proteins that are involved in a particular function of a cell. Figure 12 illustrates a module of 
the protein interaction graph described in [Bo04], comprising the complex of proteins that de- 
grade RNA in human cells. To learn more about protein interaction graphs, see [Bo04], [Ne10], 
and [Hu07]. < 


Semantic Networks Graph models are used extensively in natural language understanding 
and in information retrieval. Natural language understanding (NLU) is the subject of en- 
abling machines to disassemble and parse human speech. Its goal is to allow machines to un- 
derstand and communicate as humans do. Information retrieval (IR) is the subject of ob- 
taining information from a collection of sources based on various types of searches. Natural 
language understanding is the enabling technology when we converse with automated customer 
service agents. Advances in NLU are evident as communication between humans and machines 
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FIGURE 11 A niche overlap graph. FIGURE 12 A module of a protein interaction graph. 


EXAMPLE 13 


continually improves. When we do web searches, we take advantage of the many advances in 
information retrieval made in recent decades. 

In graph models for NLU and IR applications, vertices often represent words, phrases, or 
sentences, and edges represent connections relating to the meaning of these objects. 


In semantic networks, vertices are used to represent words, and undirected edges are used to 
connect vertices when a semantic relation holds between these words. A semantic relation is a 
relationship between two or more words that is based on the meaning of the words. For example, 
we can build a graph in which vertices represent nouns and two vertices are connected when they 
have similar meaning. For instance, the names of different countries have similar meaning, as do 
the names of different vegetables. To determine which nouns have similar meaning, large bodies 
of text can be examined. Nouns in the text that are separated by a conjunction (such as “or” or 
“and”) or a comma, or that appear in lists, are assumed to have similar meaning. For example, 
examining books on agriculture, we can determine that nouns representing names of fruits, such 
as avocado, breadfruit, guava, mango, papaya, and soursop, have similar meaning. Researchers 
who took this approach using the British National Corpus, a collection of English texts with 
100,000,000 words, produced a graph with close to 100,000 vertices, representing nouns, and 
500,000 links, connecting vertices representing pairs of words with similar meaning. Figure 13 
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FIGURE 13 A semantic network of nouns with similar meaning centered of the word 
mouse. 
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displays a small graph where the vertices represent nouns and edges connect words with similar 
meaning. This graph is centered around the word mouse. The graph illustrates that there are 
two distinct meanings for mouse. It can refer to an animal or it can refer to computer hardware. 
When a NLU program encounters the word mouse in a sentence, it can see which words with 
similar meaning would fit the sentence to help determine whether mouse refers to an animal or 
computer hardware in that sentence. < 


TOURNAMENTS We now give some examples that show how graphs can also be used to 
model different kinds of tournaments. 


EXAMPLE 14  Round-Robin Tournaments A tournament where each team plays every other team exactly 
once and no ties are allowed is called a round-robin tournament. Such tournaments can be 
modeled using directed graphs where each team is represented by a vertex. Note that (a, b) is 
an edge if team a beats team b. This graph is a simple directed graph, containing no loops or 
multiple directed edges (because no two teams play each other more than once). Such a directed 
graph model is presented in Figure 14. We see that Team 1 is undefeated in this tournament, 
and Team 3 is winless. < 

Stanford Game winners shown in blue Gonnecieut 
Georgia Iowa State 
Stanford Connecticut 
Team Team 
1 2 Florida State 
Xavier Florida State 
Gonzaga Stanford Connecticut Connecticut Mississippi State 
Team Team Oban Oklahoma Stanford Having 
6 3 Notre Dame Tennessee 
Oklahoma 
Team Team Kentucky 
$ 4 Nebraska San Diego State 
FIGURE 14 A graph FIGURE 15 A single-elimination tournament. 
model of a round-robin 
tournament. 

EXAMPLE 15 Single-Elimination Tournaments A tournament where each contestant is eliminated after 
one loss is called a single-elimination tournament. Single-elimination tournaments are often 
used in sports, including tennis championships and the yearly NCAA basketball championship. 
We can model such a tournament using a vertex to represent each game and a directed 
edge to connect a game to the next game the winner of this game played in. The graph in 
Figure 15 represents the games played by the final 16 teams in the 2010 NCAA women’s 
basketball tournament. < 

Exercises 
1. Draw graph models, stating the type of graph (from flights from Washington to Newark, and one flight from 

Table 1) used, to represent airline routes where every Washington to Miami, with 

day there are four flights from Boston to Newark, two a) an edge between vertices representing cities that have 

flights from Newark to Boston, three flights from Newark a flight between them (in either direction). 

to Miami, two flights from Miami to Newark, one flight 

from Newark to Detroit, two flights from Detroit to b) an edge between vertices representing cities for each 


Newark, three flights from Newark to Washington, two flight that operates between them (in either direction). 


c) an edge between vertices representing cities for each 
flight that operates between them (in either direction), 
plus a loop for a special sightseeing trip that takes off 


10. 
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For each undirected graph in Exercises 3-9 that is not 
simple, find a set of edges to remove to make it simple. 


ane ee 11. Let G be a simple graph. Show that the relation R on the 
and lands in Miami. i j : Í 
set of vertices of G such that uRv if and only if there is 
d) an edge from a vertex representing a city where a an edge associated to {u, v} is a symmetric, irreflexive 
flight starts to the vertex representing the city where relation on G. 
teas: 12. Let G be an undirected graph with a loop at every vertex. 
e) an edge for each flight from a vertex representing a Show that the relation R on the set of vertices of G such 
city where the flight begins to the vertex representing that uRv if and only if there is an edge associated to {u, v} 
the city where the flight ends. is a symmetric, reflexive relation on G. 
13. The intersection graph of a collection of sets Aj, 


2. What kind of graph (from Table 1) can be used to model 


a highway system between major cities where 


a) there is an edge between the vertices representing 
cities if there is an interstate highway between them? 


b) there is an edge between the vertices representing 
cities for each interstate highway between them? 


A), ..., A, is the graph that has a vertex for each of these 
sets and has an edge connecting the vertices represent- 
ing two sets if these sets have a nonempty intersection. 
Construct the intersection graph of these collections of 
sets. 


{ 
{ 
c) there is an edge between the vertices representing A; = {0, 1,8, 9} 
cities for each interstate highway between them, and 
; : sits b) A, = {..., —4, -3, —2, -1, 0}, 
there is a loop at the vertex representing a city if there 
is an interstate highway that circles this city? Ay ={...,-2,-1,0, 1,2, ...}, 
For Exercises 3-9, determine whether the graph shown has A3 = hae =6, —4, —2, 0, 2, Oa 
directed or undirected edges, whether it has multiple edges, Ay = {a 5 3, 1, 3,5, 
and whether it has one or more loops. Use your answers to As = {..., —6, —3, 0,3, 6, ...} 


determine the type of graph in Table 1 this graph is. c) A = {x|x <0}, 
A, = {x|-1<x <0}, 


3.a b A, ={x|O0<x<1}, 
Ag = {x|-1<x< 1}, 
A; = {x|x>-—-1}, 
As =R 
14. Use the niche overlap graph in Figure 11 to determine the 
x d species that compete with hawks. 
5. 15. Construct a niche overlap graph for six species of birds, 


where the hermit thrush competes with the robin and 
with the blue jay, the robin also competes with the 
mockingbird, the mockingbird also competes with the 
blue jay, and the nuthatch competes with the hairy wood- 
pecker. 


16. Draw the acquaintanceship graph that represents that 
Tom and Patricia, Tom and Hope, Tom and Sandy, Tom 
and Amy, Tom and Marika, Jeff and Patricia, Jeff and 
Mary, Patricia and Hope, Amy and Hope, and Amy and 
Marika know each other, but none of the other pairs of 
people listed know each other. 


17. We can use a graph to represent whether two people were 
alive at the same time. Draw such a graph to represent 
whether each pair of the mathematicians and computer 
scientists with biographies in the first five chapters of this 
book who died before 1900 were contemporaneous. (As- 
sume two people lived at the same time if they were alive 
during the same year.) 


4.a — b 
c d 
6. a b 
E 
e 
G d 
8. a e 
d 
b c 


18. Who can influence Fred and whom can Fred influence in 
the influence graph in Example 2? 
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19. Construct an influence graph for the board members of 
a company if the President can influence the Director of 
Research and Development, the Director of Marketing, 
and the Director of Operations; the Director of Research 
and Development can influence the Director of Opera- 
tions; the Director of Marketing can influence the Direc- 
tor of Operations; and no one can influence, or be influ- 
enced by, the Chief Financial Officer. 


20. The word apple can refer to a plant, a food, or a com- 
puter company. Construct a word graph for these nouns: 
apple, strawberry, lenovo, cheese, chocolate, ibm, oak, 
microsoft, hedge, grass, cake, quiche, hp, cider, donut, 
azalea, pine, dell, fir, raspberry. Connect two vertices by 
an undirected edge if the nouns they represent have sim- 
ilar meaning. 


21. The word rock can refer to a type of music or to some- 
thing from a mountain. Construct a word graph for 
these nouns: rock, boulder, jazz, limestone, gravel, folk, 
bachata, pumice, granite, tango, klezmer, slate, shale, 
classical, pebbles, sand, rap, marble. Connect two ver- 
tices by an undirected edge if the nouns they represent 
have similar meaning. 


22. Which other teams did Team 4 beat and which teams beat 
Team 4 in the round-robin tournament represented by the 
graph in Figure 14? 


23. In a round-robin tournament the Tigers beat the Blue 
Jays, the Tigers beat the Cardinals, the Tigers beat the 
Orioles, the Blue Jays beat the Cardinals, the Blue Jays 
beat the Orioles, and the Cardinals beat the Orioles. 
Model this outcome with a directed graph. 


24. Construct the call graph for a set of seven telephone 
numbers 555-0011, 555-1221, 555-1333, 555-8888, 
555-2222, 555-0091, and 555-1200 if there were three 
calls from 555-0011 to 555-8888 and two calls from 
555-8888 to 555-0011, two calls from 555-2222 to 
555-0091, two calls from 555-1221 to each of the 
other numbers, and one call from 555-1333 to each of 
555-0011, 555-1221, and 555-1200. 


25. Explain how the two telephone call graphs for calls made 
during the month of January and calls made during the 
month of February can be used to determine the new tele- 
phone numbers of people who have changed their tele- 
phone numbers. 


26. a) Explain how graphs can be used to model electronic 
mail messages in a network. Should the edges be di- 
rected or undirected? Should multiple edges be al- 
lowed? Should loops be allowed? 


b) Describe a graph that models the electronic mail sent 
in a network in a particular week. 


27. How can a graph that models e-mail messages sent in 
a network be used to find people who have recently 
changed their primary e-mail address? 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


36. 


37. 


38. 


How can a graph that models e-mail messages sent in 
a network be used to find electronic mail mailing lists 
used to send the same message to many different e-mail 
addresses? 


Describe a graph model that represents whether each per- 
son at a party knows the name of each other person at the 
party. Should the edges be directed or undirected? Should 
multiple edges be allowed? Should loops be allowed? 


Describe a graph model that represents a subway system 
in a large city. Should edges be directed or undirected? 
Should multiple edges be allowed? Should loops be al- 
lowed? 


For each course at a university, there may be one or more 
other courses that are its prerequisites. How can a graph 
be used to model these courses and which courses are pre- 
requisites for which courses? Should edges be directed 
or undirected? Looking at the graph model, how can we 
find courses that do not have any prerequisites and how 
can we find courses that are not the prerequisite for any 
other courses? 


Describe a graph model that represents the positive rec- 
ommendations of movie critics, using vertices to repre- 
sent both these critics and all movies that are currently 
being shown. 


Describe a graph model that represents traditional mar- 
riages between men and women. Does this graph have 
any special properties? 


Which statements must be executed before Sẹ is executed 
in the program in Example 8? (Use the precedence graph 
in Figure 10.) 


Construct a precedence graph for the following program: 


Syix:=0 
Soixi=xt1 
S3: y :=2 
Sgizi=y 
Ssixt=x+2 
S6: y =X +Z 
S7: z := 4 


Describe a discrete structure based on a graph that can 
be used to model airline routes and their flight times. 
[Hint: Add structure to a directed graph.] 


Describe a discrete structure based on a graph that can 
be used to model relationships between pairs of individ- 
uals in a group, where each individual may either like, 
dislike, or be neutral about another individual, and the 
reverse relationship may be different. [Hint: Add struc- 
ture to a directed graph. Treat separately the edges in 
opposite directions between vertices representing two 
individuals. ] 


Describe a graph model that can be used to represent all 
forms of electronic communication between two people 
in a single graph. What kind of graph is needed? 
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Definition 1 


Definition 2 


Definition 3 


EXAMPLE 1 


10.2.1 Introduction 


We introduce some of the basic vocabulary of graph theory in this section. We will use this vo- 
cabulary later in this chapter when we solve many different types of problems. One such problem 
involves determining whether a graph can be drawn in the plane so that no two of its edges cross. 
Another example is deciding whether there is a one-to-one correspondence between the vertices 
of two graphs that produces a one-to-one correspondence between the edges of the graphs. We 
will also introduce several important families of graphs often used as examples and in models. 
Several important applications will be described where these special types of graphs arise. 


10.2.2 Basic Terminology 


First, we give some terminology that describes the vertices and edges of undirected graphs. 


Two vertices u and v in an undirected graph G are called adjacent (or neighbors) in G if u 
and v are endpoints of an edge e of G. Such an edge e is called incident with the vertices u 
and v and e is said to connect u and v. 


We will also find useful terminology describing the set of vertices adjacent to a particular vertex 
of a graph. 


The set of all neighbors of a vertex v of G = (V, E), denoted by N(v), is called the neighbor- 
hood of v. If A is a subset of V, we denote by N(A) the set of all vertices in G that are adjacent 
to at least one vertex in A. So, N(A) = U,e4 NO). 


To keep track of how many edges are incident to a vertex, we make the following definition. 


The degree of a vertex in an undirected graph is the number of edges incident with it, ex- 
cept that a loop at a vertex contributes twice to the degree of that vertex. The degree of the 
vertex v is denoted by deg(v). 


What are the degrees and what are the neighborhoods of the vertices in the graphs G and H 
displayed in Figure 1? 


b c d a b c 
e o 
e 
a f e g e d 
G H 


FIGURE 1 The undirected graphs G and H. 
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EXAMPLE 2 


THEOREM 1 


EXAMPLE 3 


Solution: In G, deg(a) = 2, deg(b) = deg(c) = deg(f) =4, deg(d) = 1, deg(e) =3, and 
deg(g) = 0. The neighborhoods of these vertices are N(a) = {b, f}, N(b) = {a, c, e, f}, N(c) = 
{b,d, e, f}, N(d) = {c}, N(e) = {b,c f}, NO) = {a, b, c, e}, and N(g) = Ø. In H, deg(a) = 4, 
deg(b) = deg(e) = 6, deg(c) = 1, and deg(d) = 5. The neighborhoods of these vertices are 
N(a) = {b, d, e}, N(b) = {a, b, c,d, e}, N(c) = {b}, N(d) = {a, b, e}, and N(e) = {a,b,d}. <4 


A vertex of degree zero is called isolated. It follows that an isolated vertex is not adjacent 
to any vertex. Vertex g in graph G in Example 1 is isolated. A vertex is pendant if and only 
if it has degree one. Consequently, a pendant vertex is adjacent to exactly one other vertex. 
Vertex d in graph G in Example 1 is pendant. 

Examining the degrees of vertices in a graph model can provide useful information about 
the model, as Example 2 shows. 


What does the degree of a vertex in a niche overlap graph (introduced in Example 11 in Sec- 
tion 10.1) represent? Which vertices in this graph are pendant and which are isolated? Use the 
niche overlap graph shown in Figure 11 of Section 10.1 to interpret your answers. 


Solution: There is an edge between two vertices in a niche overlap graph if and only if the two 
species represented by these vertices compete. Hence, the degree of a vertex in a niche overlap 
graph is the number of species in the ecosystem that compete with the species represented by 
this vertex. A vertex is pendant if the species competes with exactly one other species in the 
ecosystem. Finally, the vertex representing a species is isolated if this species does not compete 
with any other species in the ecosystem. 

For instance, the degree of the vertex representing the squirrel in the niche overlap graph 
in Figure 11 in Section 10.1 is four, because the squirrel competes with four other species: the 
crow, the opossum, the raccoon, and the woodpecker. In this niche overlap graph, the mouse is 
the only species represented by a pendant vertex, because the mouse competes only with the 
shrew and all other species compete with at least two other species. There are no isolated vertices 
in the graph in this niche overlap graph because every species in this ecosystem competes with 
at least one other species. < 


What do we get when we add the degrees of all the vertices of a graph G = (V, E)? Each 
edge contributes two to the sum of the degrees of the vertices because an edge is incident with 
exactly two (possibly equal) vertices. This means that the sum of the degrees of the vertices 
is twice the number of edges. We have the result in Theorem 1, which is sometimes called the 
handshaking theorem (and is also often known as the handshaking lemma), because of the anal- 
ogy between an edge having two endpoints and a handshake involving two hands. (Exercise 6 
is based on this analogy.) 


THE HANDSHAKING THEOREM Let G = (V, E) be an undirected graph with m 


edges. Then 
2m = Ý deg(v). 
veV 


(Note that this applies even if multiple edges and loops are present.) 


How many edges are there in a graph with 10 vertices each of degree six? 


Solution: Because the sum of the degrees of the vertices is 6 - 10 = 60, it follows that 2m = 60 
where m is the number of edges. Therefore, m = 30. < 


THEOREM 2 


Definition 4 


Definition 5 


EXAMPLE 4 
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Theorem 1 shows that the sum of the degrees of the vertices of an undirected graph is even. 
This simple fact has many consequences, one of which is given as Theorem 2. 


An undirected graph has an even number of vertices of odd degree. 


Proof: Let V, and V, be the set of vertices of even degree and the set of vertices of odd degree, 
respectively, in an undirected graph G = (V, E) with m edges. Then 


2m = > deg(v) = >, deg(v) + 2 deg(v). 


veV veV; vEV, 


Because deg(v) is even for v € V}, the first term in the right-hand side of the last equality is 
even. Furthermore, the sum of the two terms on the right-hand side of the last equality is even, 
because this sum is 2m. Hence, the second term in the sum is also even. Because all the terms in 
this sum are odd, there must be an even number of such terms. Thus, there are an even number 
of vertices of odd degree. < 


Terminology for graphs with directed edges reflects the fact that edges in directed graphs 
have directions. 


When (u, v) is an edge of the graph G with directed edges, u is said to be adjacent to v and v 
is said to be adjacent from u. The vertex u is called the initial vertex of (u, v), and v is called 
the terminal or end vertex of (u, v). The initial vertex and terminal vertex of a loop are the 
same. 


Because the edges in graphs with directed edges are ordered pairs, the definition of the degree 
of a vertex can be refined to reflect the number of edges with this vertex as the initial vertex and 
as the terminal vertex. 


In a graph with directed edges the in-degree of a vertex v, denoted by deg’ (v), is the number 
of edges with v as their terminal vertex. The out-degree of v, denoted by deg*(v), is the 
number of edges with v as their initial vertex. (Note that a loop at a vertex contributes 1 to 
both the in-degree and the out-degree of this vertex.) 


Find the in-degree and out-degree of each vertex in the graph G with directed edges shown in 
Figure 2. 


FIGURE 2 The directed graph G. 
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THEOREM 3 


EXAMPLE 5 


K K K, K; 


Solution: The in-degrees in G are deg (a) = 2, deg (b) = 2, deg (c) = 3, deg (d) = 2, 
deg™ (e) = 3, and deg (f) = 0. The out-degrees are deg’ (a) = 4, deg*(b) = 1, deg*(c) = 2, 
deg*(d) = 2, deg*(e) = 3, and deg*(f) = 0. < 


Because each edge has an initial vertex and a terminal vertex, the sum of the in-degrees and 
the sum of the out-degrees of all vertices in a graph with directed edges are the same. Both of 
these sums are the number of edges in the graph. This result is stated as Theorem 3. 


Let G = (V, E) be a graph with directed edges. Then 


2 deg (v) = )' deg*(v) = [EI 


veV vEeV 


There are many properties of a graph with directed edges that do not depend on the direction 
of its edges. Consequently, it is often useful to ignore these directions. The undirected graph that 
results from ignoring directions of edges is called the underlying undirected graph. A graph 
with directed edges and its underlying undirected graph have the same number of edges. 


10.2.3 Some Special Simple Graphs 


We will now introduce several classes of simple graphs. These graphs are often used as examples 
and arise in many applications. 


Complete Graphs A complete graph on n vertices, denoted by K,, is a simple graph 
that contains exactly one edge between each pair of distinct vertices. The graphs K,,, for 


n = 1, 2,3, 4, 5, 6, are displayed in Figure 3. A simple graph for which there is at least one pair 
of distinct vertex not connected by an edge is called noncomplete. < 


FIGURE 3 The graphs K,, for 1 <n < 6. 


EXAMPLE 6 


Ke 
Cycles A cycle C,, n > 3, consists of n vertices v,, Vz, ..., v, and edges {v}, Vy}, {Vz V3}, <--> 
{v,-1¥,}, and {v,, vı}. The cycles C3, Cy, Cs, and Ce are displayed in Figure 4. < 


AOOO 


C 


FIGURE 4 The cycles C3, Cy, C5, and C,. 
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EXAMPLE 8 


Links > 


Definition 6 
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Wheels We obtain a wheel W,, when we add an additional vertex to a cycle C,,, for n > 3, and 


connect this new vertex to each of the n vertices in C,,, by new edges. The wheels W3, W,, Ws, 
and W, are displayed in Figure 5. 


AK E & 


FIGURE 5 The wheels W3, W4, W;, and W,. 


n-Cubes An n-dimensional hypercube, or n-cube, denoted by Q,,, is a graph that has vertices 
representing the 2” bit strings of length n. Two vertices are adjacent if and only if the bit strings 
that they represent differ in exactly one bit position. We display Q,, Q,, and Q; in Figure 6. 


110 111 


00 ol 


Qi Q2 Q3 


FIGURE 6 Then-cube Q,, n = 1, 2, 3. 


m 

Note that you can construct the (n + 1)-cube Q,,,, from the n-cube Q, by making two copies 
of Q,,, prefacing the labels on the vertices with a 0 in one copy of Q,„ and with a 1 in the other 
copy of Q,,, and adding edges connecting two vertices that have labels differing only in the first 
bit. In Figure 6, Q} is constructed from Q, by drawing two copies of Q, as the top and bottom 
faces of Q}, adding 0 at the beginning of the label of each vertex in the bottom face and 1 at 
the beginning of the label of each vertex in the top face. (Here, by face we mean a face of a 
cube in three-dimensional space. Think of drawing the graph Q, in three-dimensional space 
with copies of Q, as the top and bottom faces of a cube and then drawing the projection of the 
resulting depiction in the plane.) < 


10.2.4 Bipartite Graphs 


Sometimes a graph has the property that its vertex set can be divided into two disjoint subsets 
such that each edge connects a vertex in one of these subsets to a vertex in the other subset. 
For example, consider the graph representing marriages between men and women in a village, 
where each person is represented by a vertex and a marriage is represented by an edge. In this 
graph, each edge connects a vertex in the subset of vertices representing males and a vertex in 
the subset of vertices representing females. This leads us to Definition 5. 


A simple graph G is called bipartite if its vertex set V can be partitioned into two disjoint 
sets V, and V, such that every edge in the graph connects a vertex in V, and a vertex in V, 
(so that no edge in G connects either two vertices in V, or two vertices in V,). When this 
condition holds, we call the pair (V,, V3) a bipartition of the vertex set V of G. 
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EXAMPLE 9 


EXAMPLE 10 


EXAMPLE 11 


In Example 9 we will show that C, is bipartite, and in Example 10 we will show that K; is 
not bipartite. 


Cg is bipartite, as shown in Figure 7, because its vertex set can be partitioned into the two sets 
Vi = {vi v3, v5} and V, = {V9, vy, Ve }, and every edge of C, connects a vertex in V} and a vertex 
in V3. < 


K; is not bipartite. To verify this, note that if we divide the vertex set of K, into two disjoint 
sets, one of the two sets must contain two vertices. If the graph were bipartite, these two vertices 
could not be connected by an edge, but in K, each vertex is connected to every other vertex by 
an edge. < 


Are the graphs G and H displayed in Figure 8 bipartite? 


a b a b 
BLOKE 
f c 
f 
e d e d 


G H 


FIGURE7 Showing that C, is FIGURE 8 The undirected graphs G and H. 


bipartite. 


THEOREM 4 


Solution: Graph G is bipartite because its vertex set is the union of two disjoint sets, {a, b, d} 
and {c, e, f, g}, and each edge connects a vertex in one of these subsets to a vertex in the other 
subset. (Note that for G to be bipartite it is not necessary that every vertex in {a, b, d} be adjacent 
to every vertex in {c, e, f, g}. For instance, b and g are not adjacent.) 

Graph H is not bipartite because its vertex set cannot be partitioned into two subsets so 
that edges do not connect two vertices from the same subset. (The reader should verify this by 
considering the vertices a, b, and f.) < 


Theorem 4 provides a useful criterion for determining whether a graph is bipartite. 


A simple graph is bipartite if and only if it is possible to assign one of two different colors to 
each vertex of the graph so that no two adjacent vertices are assigned the same color. 


Proof: First, suppose that G = (V, E) is a bipartite simple graph. Then V = V; U V3, where V; 
and V, are disjoint sets and every edge in E connects a vertex in V, and a vertex in V}. If we 
assign one color to each vertex in V, and a second color to each vertex in V,, then no two 
adjacent vertices are assigned the same color. 

Now suppose that it is possible to assign colors to the vertices of the graph using just two 
colors so that no two adjacent vertices are assigned the same color. Let V, be the set of vertices 
assigned one color and V, be the set of vertices assigned the other color. Then, V, and V, 
are disjoint and V = V; U V>. Furthermore, every edge connects a vertex in V, and a vertex 
in V, because no two adjacent vertices are either both in V} or both in V,. Consequently, G 
is bipartite. < 


EXAMPLE 12 


EXAMPLE 13 
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We illustrate how Theorem 4 can be used to determine whether a graph is bipartite in 
Example 12. 


Use Theorem 4 to determine whether the graphs in Example 11 are bipartite. 


Solution: We first consider the graph G. We will try to assign one of two colors, say red and 
blue, to each vertex in G so that no edge in G connects a red vertex and a blue vertex. Without 
loss of generality we begin by arbitrarily assigning red to a. Then, we must assign blue to c, e, 
f, and g, because each of these vertices is adjacent to a. To avoid having an edge with two blue 
endpoints, we must assign red to all the vertices adjacent to either c, e, f, or g. This means that 
we must assign red to both b and d (and means that a must be assigned red, which it already has 
been). We have now assigned colors to all vertices, with a, b, and d red and c, e, f, and g blue. 
Checking all edges, we see that every edge connects a red vertex and a blue vertex. Hence, by 
Theorem 4 the graph G is bipartite. 

Next, we will try to assign either red or blue to each vertex in H so that no edge in H connects 
ared vertex and a blue vertex. Without loss of generality we arbitrarily assign red to a. Then, we 
must assign blue to b, e, and f, because each is adjacent to a. But this is not possible because e 
and f are adjacent, so both cannot be assigned blue. This argument shows that we cannot assign 
one of two colors to each of the vertices of H so that no adjacent vertices are assigned the same 
color. It follows by Theorem 4 that H is not bipartite. < 


Theorem 4 is an example of a result in the part of graph theory known as graph colorings. 
Graph colorings is an important part of graph theory with important applications. We will study 
graph colorings further in Section 10.8. 

Another useful criterion for determining whether a graph is bipartite is based on the notion 
of a path, a topic we study in Section 10.4. A graph is bipartite if and only if it is not possible 
to start at a vertex and return to this vertex by traversing an odd number of distinct edges. We 
will make this notion more precise when we discuss paths and circuits in graphs in Section 10.4 
(see Exercise 63 in that section). 


Complete Bipartite Graphs A complete bipartite graph K,, ,, is a graph that has its vertex 


set partitioned into two subsets of m and n vertices, respectively with an edge between two 
vertices if and only if one vertex is in the first subset and the other vertex is in the second 
subset. The complete bipartite graphs K, 3, K3 3, K3 5, and K, ẹ are displayed in Figure 9. < 


PA PA 


23 3,3 


3,5 2,6 


FIGURE 9 Some complete bipartite graphs. 
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10.2.5 Bipartite Graphs and Matchings 


Bipartite graphs can be used to model many types of applications that involve matching the 
elements of one set to elements of another, as Example 14 illustrates. 


Job Assignments Suppose that there are m employees in a group and n different jobs that 
need to be done, where m > n. Each employee is trained to do one or more of these n jobs. We 
would like to assign an employee to each job. To help with this task, we can use a graph to model 
employee capabilities. We represent each employee by a vertex and each job by a vertex. For 
each employee, we include an edge from that employee to all jobs that the employee has been 
trained to do. Note that the vertex set of this graph can be partitioned into two disjoint sets, the 
set of employees and the set of jobs, and each edge connects an employee to a job. Consequently, 
this graph is bipartite, where the bipartition is (E, J) where E is the set of employees and J is 
the set of jobs. We now consider two different scenarios. 

First, suppose that a group has four employees: Alvarez, Berkowitz, Chen, and Davis; 
and suppose that four jobs need to be done to complete Project 1: requirements, architecture, 
implementation, and testing. Suppose that Alvarez has been trained to do requirements and 
testing; Berkowitz has been trained to do architecture, implementation, and testing; Chen has 
been trained to do requirements, architecture, and implementation; and Davis has only been 
trained to do requirements. We model these employee capabilities using the bipartite graph in 
Figure 10(a). 


Alvarez Berkowitz Chen Davis Washington Xuan Ybarra Ziegler 
requirements architecture implementation testing requirements architecture implementation testing 


(a) (b) 
FIGURE 10 Modeling the jobs for which employees have been trained. 


Second, suppose that a second group also has four employees: Washington, Xuan, Ybarra, 
and Ziegler; and suppose that the same four jobs need to be done to complete Project 2 as 
are needed to complete Project 1. Suppose that Washington has been trained to do architecture; 
Xuan has been trained to do requirements, implementation, and testing; Ybarra has been trained 
to do architecture; and Ziegler has been trained to do requirements, architecture and testing. We 
model these employee capabilities using the bipartite graph in Figure 10(b). 

To complete Project 1, we must assign an employee to each job so that every job has an 
employee assigned to it, and so that no employee is assigned more than one job. We can do this 
by assigning Alvarez to testing, Berkowitz to implementation, Chen to architecture, and Davis 
to requirements, as shown in Figure 10(a) (where blue lines show this assignment of jobs). 

To complete Project 2, we must also assign an employee to each job so that every job has 
an employee assigned to it and no employee is assigned more than one job. However, this is 
impossible because there are only two employees, Xuan and Ziegler, who have been trained for 
at least one of the three jobs of requirements, implementation, and testing. Consequently, there 
is no way to assign three different employees to these three jobs so that each job is assigned an 
employee with the appropriate training. < 


Finding an assignment of jobs to employees can be thought of as finding a matching in 
the graph model, where a matching M in a simple graph G = (V, E) is a subset of the set E of 
edges of the graph such that no two edges are incident with the same vertex. In other words, a 
matching is a subset of edges such that if {s, t} and {u, v} are distinct edges of the matching, 


EXAMPLE 15 


Hall’s marriage theorem 
is an example of a 
theorem where obvious 
necessary conditions are 
sufficient too. 


THEOREM 5 


Courtesy of the Edinburgh 
Mathematical Society 
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then s, t, u, and v are distinct. A vertex that is the endpoint of an edge of a matching M is said to 
be matched in M; otherwise it is said to be unmatched. A maximum matching is a matching 
with the largest number of edges. We say that a matching M in a bipartite graph G = (V, E) 
with bipartition (V,, V2) is a complete matching from V, to V, if every vertex in V} is the 
endpoint of an edge in the matching, or equivalently, if |M| = |V,|. For example, to assign jobs 
to employees so that the largest number of jobs are assigned employees, we seek a maximum 
matching in the graph that models employee capabilities. To assign employees to all jobs we 
seek a complete matching from the set of jobs to the set of employees. In Example 14, we found 
a complete matching from the set of jobs to the set of employees for Project 1, and this matching 
is a maximum matching, and we showed that no complete matching exists from the set of jobs 
to the employees for Project 2. 
We now give an example of how matchings can be used to model marriages. 


Marriages onan Island Suppose that there are m men and n women on an island. Each person 
has a list of members of the opposite gender acceptable as a spouse. We construct a bipartite 
graph G = (V,, V2) where V; is the set of men and V, is the set of women so that there is an edge 
between a man and a woman if they find each other acceptable as a spouse. A matching in this 
graph consists of a set of edges, where each pair of endpoints of an edge is a husband-wife pair. 
A maximum matching is a largest possible set of married couples, and a complete matching of 
V, is a set of married couples where every man is married, but possibly not all women. < 


NECESSARY AND SUFFICIENT CONDITIONS FOR COMPLETE MATCHINGS We 
now turn our attention to the question of determining whether a complete matching from V, 
to V, exists when (V,, V2) is a bipartition of a bipartite graph G = (V, E). We will introduce a 
theorem that provides a set of necessary and sufficient conditions for the existence of a complete 
matching. This theorem was proved by Philip Hall in 1935. 


HALL’S MARRIAGE THEOREM The bipartite graph G = (V, E) with bipartition 
(Vi V2) has a complete matching from V, to V, if and only if |N(A)| > |A] for all subsets 
A of V,. 


Proof: We first prove the only if part of the theorem. To do so, suppose that there is a complete 
matching M from V, to V,. Then, if A C V}, for every vertex v € A, there is an edge in M 
connecting v to a vertex in V,. Consequently, there are at least as many vertices in V, that are 
neighbors of vertices in V} as there are vertices in V}. It follows that |N(A)| > |A]. 


PHILIP HALL (1904-1982) Philip Hall grew up in London, where his mother was a dressmaker. He won a 
scholarship for board school reserved for needy children, and later a scholarship to King’s College of Cambridge 
University. He received his bachelors degree in 1925. In 1926, unsure of his career goals, he took a civil service 
exam, but decided to continue his studies at Cambridge after failing. 

In 1927 Hall was elected to a fellowship at King’s College; soon after, he made his first important 
discovery in group theory. The results he proved are now known as Hall’s theorems. In 1933 he was ap- 
pointed as a Lecturer at Cambridge, where he remained until 1941. During World War II he worked as 
a cryptographer at Bletchley Park breaking Italian and Japanese codes. At the end of the war, Hall re- 
turned to King’s College, and was soon promoted. In 1953 he was appointed to the Sadleirian Chair. 
His work during the 1950s proved to be extremely influential to the rapid development of group theory 
during the 1960s. 


Hall loved poetry and recited it beautifully in Italian and Japanese, as well as English. He was interested in art, music, and 
botany. He was quite shy and disliked large groups of people. Hall had an incredibly broad and varied knowledge, and was respected 
for his integrity, intellectual standards, and judgement. He was beloved by his students. 
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To prove the if part of the theorem, the more difficult part, we need to show that if 
|N(A)| = |A] for all A C V,, then there is a complete matching M from V, to V}. We will use 
strong induction on |V,| to prove this. 


Basis step: If |V,| = 1, then V} contains a single vertex vy. Because |N({vo ÐI = |{vo}] = 1, 
there is at least one edge connecting vp and a vertex wọ E€ V,. Any such edge forms a complete 
matching from V} to V}. 


Inductive step: We first state the inductive hypothesis. 


Inductive hypothesis: Let k be a positive integer. If G = (V, E) is a bipartite graph with bipartition 
(V,, V2), and |V,| =j < k, then there is a complete matching M from V, to V, whenever the 
condition that |N(A)| > |A] for all A C V; is met. 

Now suppose that H = (W, F) is a bipartite graph with bipartition (W,, W,) and |W; | = k + 
1. We will prove that the inductive holds using a proof by cases, using two case. Case (i) applies 
when for all integers j with 1 < j < k, the vertices in every set of j elements from W, are adjacent 
to at least j + 1 elements of W,. Case (ii) applies when for some j with 1 < j < k there is a subset 
Wi; of j vertices such that there are exactly j neighbors of these vertices in W}. Because either 
Case (i) or Case (ii) holds, we need only consider these cases to complete the inductive step. 


Case (i): Suppose that for all integers j with 1 < j < k, the vertices in every subset of j elements 
from W, are adjacent to at least j + 1 elements of W,. Then, we select a vertex v € W, and an 
element w E€ N({v}), which must exist by our assumption that |V({v}| > |{v}| = 1. We delete 
v and w and all edges incident to them from H. This produces a bipartite graph H’ with biparti- 
tion (W, — {v}, W» — {w}). Because |W, — {v}| = k, the inductive hypothesis tells us there is a 
complete matching from W, — {v} to W, — {w}. Adding the edge from v to w to this complete 
matching produces a complete matching from W, to W3. 


Case (ii): Suppose that for some j with 1 <j < k, there is a subset Wi of j vertices such that 
there are exactly j neighbors of these vertices in W,. Let W3 be the set of these neighbors. Then, 
by the inductive hypothesis there is a complete matching from wi to W3. Remove these 2j 
vertices from W, and W, and all incident edges to produce a bipartite graph K with bipartition 
(W; — Wi, W, — W3). 

We will show that the graph K satisfies the condition |N(A)| > |A| for all subsets A of W, — 
Wi . If not, there would be a subset of t vertices of W, — Wi where 1 < t < k + 1 — j such that the 
vertices in this subset have fewer than ż vertices of W, — W3 as neighbors. Then, the set of j + t 
vertices of W, consisting of these f vertices together with the j vertices we removed from W, has 
fewer than j + t neighbors in W,, contradicting the hypothesis that |N(A)| > |A| for all A G W,. 

Hence, by the inductive hypothesis, the graph K has a complete matching. Combining this 
complete matching with the complete matching from W; to W;, we obtain a complete matching 
from W; to W3. 


We have shown that in both cases there is a complete matching from W, to W,. This com- 
pletes the inductive step and completes the proof. < 


We have used strong induction to prove Hall’s marriage theorem. Although our proof is 
elegant, it does have some drawbacks. In particular, we cannot construct an algorithm based on 
this proof that finds a complete matching in a bipartite graph. For a constructive proof that can 
be used as the basis of an algorithm, see [Gi85]. 


10.2.6 Some Applications of Special Types of Graphs 


We conclude this section by introducing some additional graph models that involve the special 
types of graph we have discussed in this section. 
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Local Area Networks The various computers in a building, such as minicomputers and per- 
sonal computers, as well as peripheral devices such as printers and plotters, can be connected 
using a local area network. Some of these networks are based on a star topology, where all 
devices are connected to a central control device. A local area network can be represented using 
a complete bipartite graph K, „, as shown in Figure 11(a). Messages are sent from device to 
device through the central control device. 


(a) (b) (c) 


FIGURE 11 Star, ring, and hybrid topologies for local area networks. 


Other local area networks are based on a ring topology, where each device is connected to 
exactly two others. Local area networks with a ring topology are modeled using n-cycles, C,,, 
as shown in Figure 11(b). Messages are sent from device to device around the cycle until the 
intended recipient of a message is reached. 

Finally, some local area networks use a hybrid of these two topologies. Messages may be 
sent around the ring, or through a central device. This redundancy makes the network more 
reliable. Local area networks with this redundancy can be modeled using wheels W,, as shown 


n? 


in Figure 11(c). < 


Interconnection Networks for Parallel Computation For many years, computers executed 
programs one operation at a time. Consequently, the algorithms written to solve problems were 
designed to perform one step at a time; such algorithms are called serial. (Almost all algo- 
rithms described in this book are serial.) However, many computationally intense problems, 
such as weather simulations, medical imaging, and cryptanalysis, cannot be solved in a reason- 
able amount of time using serial operations, even on a supercomputer. Furthermore, there is a 
physical limit to how fast a computer can carry out basic operations, so there will always be 
problems that cannot be solved in a reasonable length of time using serial operations. 

Parallel processing, which uses computers made up of many separate processors, each 
with its own memory, helps overcome the limitations of computers with a single processor. 
Parallel algorithms, which break a problem into a number of subproblems that can be solved 
concurrently, can then be devised to rapidly solve problems using a computer with multiple 
processors. In a parallel algorithm, a single instruction stream controls the execution of the 
algorithm, sending subproblems to different processors, and directs the input and output of 
these subproblems to the appropriate processors. 

When parallel processing is used, one processor may need output generated by another 
processor. Consequently, these processors need to be interconnected. We can use the appropriate 
type of graph to represent the interconnection network of the processors in a computer with 
multiple processors. In the following discussion, we will describe the most commonly used 
types of interconnection networks for parallel processors. The type of interconnection network 
used to implement a particular parallel algorithm depends on the requirements for exchange of 
data between processors, the desired speed, and, of course, the available hardware. 

The simplest, but most expensive, network-interconnecting processors include a two-way 
link between each pair of processors. This network can be represented by K,,, the complete graph 
on n vertices, when there are n processors. However, there are serious problems with this type 
of interconnection network because the required number of connections is so large. In reality, 
the number of direct connections to a processor is limited, so when there are a large number of 
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processors, a processor cannot be linked directly to all others. For example, when there are 64 
processors, C(64, 2) = 2016 connections would be required, and each processor would have to 
be directly connected to 63 others. 

On the other hand, perhaps the simplest way to interconnect n processors is to use an ar- 
rangement known as a linear array. Each processor P,, other than P} and P,„, is connected to its 
neighbors P,_, and P,,; via a two-way link. P} is connected only to P,, and P, is connected only 
to P,,_,. The linear array for six processors is shown in Figure 12. The advantage of a linear array 
is that each processor has at most two direct connections to other processors. The disadvantage 
is that it is sometimes necessary to use a large number of intermediate links, called hops, for 
processors to share information. 

The mesh network (or two-dimensional array) is a commonly used interconnection net- 
work. In such a network, the number of processors is a perfect square, say n = m°. The n proces- 
sors are labeled P(i, j), 0 <i < m — 1, 0 < j < m — 1. Two-way links connect processor P(i, j) 
with its four neighbors, processors P(i + 1, j) and P(i, j + 1), as long as these are processors in 
the mesh. (Note that four processors, on the corners of the mesh, have only two adjacent pro- 
cessors, and other processors on the boundaries have only three neighbors. Sometimes a variant 
of a mesh network in which every processor has exactly four connections is used; see Exercise 
74.) The mesh network limits the number of links for each processor. Communication between 
some pairs of processors requires O( yn) = O(m) intermediate links. (See Exercise 75.) The 
graph representing the mesh network for 16 processors is shown in Figure 13. 

One important type of interconnection network is the hypercube. For such a network, the 
number of processors is a power of 2, n = 2”. The n processors are labeled Po, Pi, ..., P,_1. 
Each processor has two-way connections to m other processors. Processor P; is linked to the 
processors with indices whose binary representations differ from the binary representation 
of i in exactly one bit. The hypercube network balances the number of direct connections for 
each processor and the number of intermediate connections required so that processors can 
communicate. Many computers have been built using a hypercube network, and many paral- 
lel algorithms have been devised that use a hypercube network. The graph Q,,,, the m-cube, 
represents the hypercube network with n = 2” processors. Figure 14 displays the hypercube 
network for eight processors. (Figure 14 displays a different way to draw Q, than was shown in 
Figure 6.) < 


10.2.7 New Graphs from Old 


Sometimes we need only part of a graph to solve a problem. For instance, we may care only 
about the part of a large computer network that involves the computer centers in New York, 
Denver, Detroit, and Atlanta. Then we can ignore the other computer centers and all telephone 
lines not linking two of these specific four computer centers. In the graph model for the large 
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FIGURE 14 A hypercube network for eight processors. FIGURE 15 A subgraph of K;. 


Definition 7 


Definition 8 


EXAMPLE 18 


network, we can remove the vertices corresponding to the computer centers other than the four 
of interest, and we can remove all edges incident with a vertex that was removed. When edges 
and vertices are removed from a graph, without removing endpoints of any remaining edges, a 
smaller graph is obtained. Such a graph is called a subgraph of the original graph. 


A subgraph of a graph G = (V, E) is a graph H = (W, F), where W C V and F C E. A sub- 
graph H of Gis a proper subgraph of Gif H £ G. 


Given a set of vertices of a graph, we can form a subgraph of this graph with these vertices 
and the edges of the graph that connect them. 


Let G = (V, E) be a simple graph. The subgraph induced by a subset W of the vertex set V 
is the graph (W, F), where the edge set F contains an edge in E if and only if both endpoints 
of this edge are in W. 


The graph G shown in Figure 15 is a subgraph of K5. If we add the edge connecting c and e to 
G, we obtain the subgraph induced by W = {a, b, c, e}. < 


REMOVING OR ADDING EDGES OF A GRAPH Given a graph G = (V, E) and an edge 
e E E, we can produce a subgraph of G by removing the edge e. The resulting subgraph, denoted 
by G — e, has the same vertex set V as G. Its edge set is E — {e}. Hence, 


G-e=(V,E - f{e}). 


Similarly, if E’ is a subset of E, we can produce a subgraph of G by removing the edges in £’ 
from the graph. The resulting subgraph has the same vertex set V as G. Its edge set is E — E’. 

We can also add an edge e to a graph to produce a new larger graph when this edge connects 
two vertices already in G. We denote by G + e the new graph produced by adding a new edge e, 
connecting two previously nonincident vertices, to the graph G. Hence, 


Gte=(V,EU {e}). 


The vertex set of G + e is the same as the vertex set of G and the edge set is the union of the 
edge set of G and the set {e}. (See Example19 for examples of removing an edge from a graph 
and adding an edge to a graph.) 


EDGE CONTRACTIONS Sometimes when we remove an edge from a graph, we do not want 
to retain the endpoints of this edge as separate vertices in the resulting subgraph. In such a case 
we perform an edge contraction, which removes an edge e with endpoints u and v and merges u 
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EXAMPLE 19 


and w into a new single vertex w, and for each edge with u or v as an endpoint replaces the edge 
with one with w as endpoint in place of u or v and with the same second endpoint. Hence, the 
contraction of the edge e with endpoints u and v in the graph G = (V, E) produces a new graph 
G’ = (V', E’) (which is not a subgraph of G), where V’ = V — {u, v} U {w} and E’ contains the 
edges in E which do not have either u or v as endpoints and an edge connecting w to every 
neighbor of either u or v in V. For example, the contraction of the edge connecting the vertices 
e and c in the graph G; in Figure 16 produces a new graph G{ with vertices a, b, d, and w. As 
in G4, there is an edge in G| connecting a and b and an edge connecting a and d. There also is 
an edge in G’ that connects b and w that replaces the edges connecting b and c and connecting 
b and e in G, and an edge in G} that connects d and w replacing the edge connecting d and e in 
G,. (Also, see Example 19 for an example of the contraction of an edge in a graph.) 


REMOVING VERTICES FROM A GRAPH When we remove a vertex v and all edges 
incident to it from G = (V, E), we produce a subgraph, denoted by G — v. Observe that 
G-v=(V-— {v}, E’), where E’ is the set of edges of G not incident to v. Similarly, if V’ is a 
subset of V, then the graph G — V’ is the subgraph (V — V’, E’), where E’ is the set of edges of G 
not incident to a vertex in V’. (See Example 19 for an example of the removal of a vertex from a 
graph.) 


Figure 16 displays an undirected graph G with four different graphs that are the result of different 
operations on G. These are: 


(a) G — {b,c}, constructed from G by removing the edge {b, c} 

(b) G+ {e, d}, constructed from G by adding the edge {e, d} 

(c) the contraction of G, constructed from G by replacing the edge {b, c} with a new vertex f, 
and replacing the edges {c, d}, {a, b}, {b, e}, and {c, e} with the new edges {a,f}, {f d}, 
and {f e} 

(d) G — c, constructed from G by removing the vertex c and the edges {b, c}, {c, d} and {c, e} 

< 


GRAPH UNIONS Two or more graphs can be combined in various ways. The new graph that 
contains all the vertices and edges of these graphs is called the union of the graphs. We will 
give a more formal definition for the union of two simple graphs. 


a 
> 
fay 

ean 
a 
> 
a 

ea 
a 
> 
fy 
a 


e e e 
G G- {b,c} G + {e,d} 
(a) (b) 
a f d a b 
| e 
G contracted by replacing G-c 
{b, c} by f 
(c) (d) 


FIGURE 16 The graph G and four graphs resulting from different operations on G. 
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FIGURE 17 (a) The simple graphs G, and G,. (b) Their union G} U G,. 


Definition 9 The union of two simple graphs G, = (V,, E,) and G, = (V,, E>) is the simple graph with 
vertex set V; U V, and edge set E, U E,. The union of G, and G, is denoted by G, UG). 
EXAMPLE 20 Find the union of the graphs G, and G, shown in Figure 17(a). < 
Solution: The vertex set of the union G} U G, is the union of the two vertex sets, namely, 
{a, b, c, d, e, f }. The edge set of the union is the union of the two edge sets. The union is displayed 
in Figure 17(b). 
Exercises 


In Exercises 1—3 find the number of vertices, the number of 
edges, and the degree of each vertex in the given undirected 
graph. Identify all isolated and pendant vertices. 


l. a b ĉ 
© 


f i h g e 
4. Find the sum of the degrees of the vertices of each graph 
in Exercises 1-3 and verify that it equals twice the num- 
ber of edges in the graph. 


5. Can a simple graph exist with 15 vertices each of degree 
five? 


6. Show that the sum, over the set of people at a party, of 
the number of people a person has shaken hands with, is 
even. Assume that no one shakes his or her own hand. 

In Exercises 7—9 determine the number of vertices and edges 
and find the in-degree and out-degree of each vertex for the 
given directed multigraph. 


7. b 8. a b 


d c d c 
9, a b 
e 
d c e 


10. For each of the graphs in Exercises 7—9 determine the 
sum of the in-degrees of the vertices and the sum of the 
out-degrees of the vertices directly. Show that they are 
both equal to the number of edges in the graph. 
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12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 
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Construct the underlying undirected graph for the graph 
with directed edges in Figure 2. 


What does the degree of a vertex represent in the acquain- 
tanceship graph, where vertices represent all the people 
in the world? What does the neighborhood of a vertex in 
this graph represent? What do isolated and pendant ver- 
tices in this graph represent? In one study it was estimated 
that the average degree of a vertex in this graph is 1000. 
What does this mean in terms of the model? 


What does the degree of a vertex represent in an academic 
collaboration graph? What does the neighborhood of a 
vertex represent? What do isolated and pendant vertices 
represent? 


What does the degree of a vertex in the Hollywood graph 
represent? What does the neighborhood of a vertex rep- 
resent? What do the isolated and pendant vertices repre- 
sent? 


What do the in-degree and the out-degree of a vertex in a 
telephone call graph, as described in Example 4 of Sec- 
tion 10.1, represent? What does the degree of a vertex in 
the undirected version of this graph represent? 


What do the in-degree and the out-degree of a vertex in 
the web graph, as described in Example 5 of Section 10.1, 
represent? 


What do the in-degree and the out-degree of a vertex in a 
directed graph modeling a round-robin tournament rep- 
resent? 


Show that in a simple graph with at least two vertices 
there must be two vertices that have the same degree. 


Use Exercise 18 to show that in a group of people, there 
must be two people who are friends with the same num- 
ber of other people in the group. 

Draw these graphs. 
a) K 

d) C, 


b) Kis 
e) W 


c) Ky, 
f) Q, 


In Exercises 21-25 determine whether the graph is bipartite. 
You may find it useful to apply Theorem 4 and answer the 
question by determining whether it is possible to assign ei- 
ther red or blue to each vertex so that no two adjacent vertices 
are assigned the same color. 


21. 


a b 22. b c 


24. 


25. 


26. 


27. 


28. 


29. 


a b 
£ c 
e d 
b 
a c 
f d 


é 

For which values of n are these graphs bipartite? 
a) K, b) C, c) W, d) Q, 
Suppose that there are four employees in the computer 
support group of the School of Engineering of a large uni- 
versity. Each employee will be assigned to support one 
of four different areas: hardware, software, networking, 
and wireless. Suppose that Ping is qualified to support 
hardware, networking, and wireless; Quiggley is quali- 
fied to support software and networking; Ruiz is qualified 
to support networking and wireless, and Sitea is qualified 
to support hardware and software. 

a) Use a bipartite graph to model the four employees and 
their qualifications. 

b) Use Hall’s theorem to determine whether there is an 
assignment of employees to support areas so that each 
employee is assigned one area to support. 

c) Ifan assignment of employees to support areas so that 
each employee is assigned to one support area exists, 
find one. 

Suppose that a new company has five employees: 
Zamora, Agraharam, Smith, Chou, and Macintyre. Each 
employee will assume one of six responsiblities: plan- 
ning, publicity, sales, marketing, development, and in- 
dustry relations. Each employee is capable of doing one 
or more of these jobs: Zamora could do planning, sales, 
marketing, or industry relations; Agraharam could do 
planning or development; Smith could do publicity, sales, 
or industry relations; Chou could do planning, sales, 
or industry relations; and Macintyre could do planning, 
publicity, sales, or industry relations. 

a) Model the capabilities of these employees using a bi- 
partite graph. 

b) Find an assignment of responsibilites such that each 
employee is assigned one responsibility. 

c) Is the matching of responsibilities you found in part 
(b) a complete matching? Is it a maximum matching? 


Suppose that there are five young women and five young 
men on an island. Each man is willing to marry some of 
the women on the island and each woman is willing to 


30. 


marry any man who is willing to marry her. Suppose that 
Sandeep is willing to marry Tina and Vandana; Barry is 
willing to marry Tina, Xia, and Uma; Teja is willing to 
marry Tina and Zelda; Anil is willing to marry Vandana 
and Zelda; and Emilio is willing to marry Tina and Zelda. 
Use Hall’s theorem to show there is no matching of the 
young men and young women on the island such that each 
young man is matched with a young woman he is willing 
to marry. 


Suppose that there are five young women and six young 
men on an island. Each woman is willing to marry some 
of the men on the island and each man is willing to marry 
any woman who is willing to marry him. Suppose that 

Anna is willing to marry Jason, Larry, and Matt; Barbara 

is willing to marry Kevin and Larry; Carol is willing to 

marry Jason, Nick, and Oscar; Diane is willing to marry 

Jason, Larry, Nick, and Oscar; and Elizabeth is willing to 

marry Jason and Matt. 

a) Model the possible marriages on the island using a 
bipartite graph. 

b) Find a matching of the young women and the young 
men on the island such that each young woman is 
matched with a young man whom she is willing to 
marry. 

c) Is the matching you found in part (b) a complete 
matching? Is it a maximum matching? 


Each of Exercises 31-33 can be solved using Hall’s theorem. 


*31. 


*32. 


* 33. 


* 34, 


Suppose there is an integer k such that every man on a 
desert island is willing to marry exactly k of the women 
on the island and every woman on the island is willing to 
marry exactly k of the men. Also, suppose that a man is 
willing to marry a woman if and only if she is willing to 
marry him. Show that it is possible to match the men and 
women on the island so that everyone is matched with 
someone that they are willing to marry. 


Suppose that 2n tennis players compete in a round-robin 
tournament. Every player has exactly one match with ev- 
ery other player during 2n — 1 consecutive days. Every 
match has a winner and a loser. Show that it is possible 
to select a winning player each day without selecting the 
same player twice. 


Suppose that m people are selected as prize winners in a 
lottery, where each winner can select two prizes from a 
collection of different prizes. Show if there are 2m prizes 
that every winner wants, then every winner is able to 
select two prizes that they want. 


In this exercise we prove a theorem of Øystein Ore. 
Suppose that G = (V, E) is a bipartite graph with bi- 
partition (V,, V3) and that A C V,. Show that the max- 
imum number of vertices of V} that are the endpoints 
of a matching of G equals |V; | — max,cy,def(A), where 
def(A) = |A| — |N(A)|. (Here, def(A) is called the de- 
ficiency of A.) [Hint: Form a larger graph by adding 
maX,cy,def(A) new vertices to V, and connect all of 
them to the vertices of V,.] 


35. 


36. 


37. 
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For the graph G in Exercise 1 find 

a) the subgraph induced by the vertices a, b, c, and f. 

b) the new graph G} obtained from G by contracting the 
edge connecting b and f. 


Let n be a positive integer. Show that a subgraph induced 
by a nonempty subset of the vertex set of K, is a complete 
graph. 

How many vertices and how many edges do these graphs 
have? 

a) K, b) C, 
d) Koen e) Qn 


c) W, 


The degree sequence of a graph is the sequence of the de- 
grees of the vertices of the graph in nonincreasing order. For 
example, the degree sequence of the graph G in Example 1 is 
4, 4, 4, 3, 2, 1,0. 


38. 


39. 


40. 
41. 
42. 
43. 


A sequence d,, dy, .. 


Find the degree sequences for each of the graphs in Ex- 
ercises 21-25. 

Find the degree sequence of each of the following 
graphs. 

a) K, b) CG, 
d) K>; e) Q3 
What is the degree sequence of the bipartite graph K,,, , 
where m and n are positive integers? Explain your answer. 


c) W, 


What is the degree sequence of K, 
integer? Explain your answer. 


where n is a positive 
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How many edges does a graph have if its degree sequence 
is 4, 3, 3, 2, 2? Draw such a graph. 

How many edges does a graph have if its degree sequence 
is 5, 2, 2, 2, 2, 1? Draw such a graph. 

., d, is called graphic if it is the degree 


sequence of a simple graph. 


44. 


45. 


*46. 


* 47. 


* 48. 


Determine whether each of these sequences is graphic. 
For those that are, draw a graph having the given degree 
sequence. 

a) 5,4,3,2,1,0 b) 6,5,4,3,2,1 ©) 
d) 3,3,3,2,2,2 e) 3,3,2,2,2,2 f) 
g) 5,3,3,3,3,3 h) 5,5, 4,3, 2,1 
Determine whether each of these sequences is graphic. 
For those that are, draw a graph having the given degree 
sequence. 


DD Dd pdiyD 
111, 1,1,1 


> 


> 


a) 3,3,3,3,2 b) 5,4,3,2,1 c) 4,4, 3,2, 1 
d) 4,4,3,3,3 œ) 3,2,2,1,0 f) 1,1,1,1,1 
Suppose that d,, d>, ...,d,, is a graphic sequence. Show 
that there is a simple graph with vertices v,, v,..., Vv, 


such that deg(v,) = d; for i= 1, 2,...,n and v, is adja- 
cent to V5, ..., Va ais 
Show that a sequence d}, d), ...,d,, of nonnegative inte- 
gers in nonincreasing order is a graphic sequence if and 
only if the sequence obtained by reordering the terms of 
the sequence d, —1,...,dq.4; — 1,dq,49,--.,d, So that 


the terms are in nonincreasing order is a graphic se- 
quence. 

Use Exercise 47 to construct a recursive algorithm for de- 
termining whether a nonincreasing sequence of positive 
integers is graphic. 
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49. Show that every nonincreasing sequence of nonnegative 
integers with an even sum of its terms is the degree se- 
quence of a pseudograph, that is, an undirected graph 
where loops are allowed. [Hint: Construct such a graph 
by first adding as many loops as possible at each vertex. 
Then add additional edges connecting vertices of odd 
degree. Explain why this construction works. ] 

50. How many subgraphs with at least one vertex does K, 
have? 

51. How many subgraphs with at least one vertex does K3 
have? 

52. How many subgraphs with at least one vertex does W3 
have? 

53. Draw all subgraphs of this graph. 


a b 


c d 


54. Let G be a graph with v vertices and e edges. Let M be 
the maximum degree of the vertices of G, and let m be 
the minimum degree of the vertices of G. Show that 
a) 2e/v>m. b) 2e/v < M. 

A simple graph is called regular if every vertex of this graph 

has the same degree. A regular graph is called n-regular if 

every vertex in this graph has degree n. 

55. For which values of n are these graphs regular? 

a) K, b) C, c) W, d) Q, 

56. For which values of m and n is K,,,,, regular? 

57. How many vertices does a regular graph of degree four 
with 10 edges have? 

In Exercises 58—60 find the union of the given pair of simple 

graphs. (Assume edges with the same endpoints are the same.) 


58. a 


f L 
f b 6 
e c 
d d 
59. a b a f b 
e e 
c d c g d 
60. a b a e 


h 


a 
a 

Ss 

oq 


61. 


62. 


63. 


64. 


65. 


* 66. 


67. 


* 68, 


The complementary graph Gofa simple graph G has 
the same vertices as G. Two vertices are adjacent in G if 
and only if they are not adjacent in G. Describe each of 
these graphs. 


a) K, b) Kau 9G, d) Q, 


m,n 


If G is a simple graph with 15 edges and G has 13 edges, 
how many vertices does G have? 


If the simple graph G has v vertices and e edges, how 
many edges does G have? 


If the degree sequence of the simple graph G is 
4, 3, 3, 2, 2, what is the degree sequence of G? 


If the degree sequence of the simple graph G is 
di, d>, ...,d,, What is the degree sequence of G? 


s n? 


Show that if G is a bipartite simple graph with v vertices 
and e edges, then e < v? /4. 


Show that if G is a simple graph with n vertices, then the 
union of G and G is K,. 


Describe an algorithm to decide whether a graph is bipar- 
tite based on the fact that a graph is bipartite if and only 
if it is possible to color its vertices two different colors 
so that no two vertices of the same color are adjacent. 


The converse of a directed graph G = (V, E), denoted 
by G, is the directed graph (V, F), where the set F of 
edges of G°°”” is obtained by reversing the direction of each 
edge in E. 


69. 


70. 


71. 


72. 


73. 


74. 


75. 


Draw the converse of each of the graphs in Exercises 7-9 
in Section 10.1. 


Show that (G) = G whenever G is a directed 
graph. 


Show that the graph G is its own converse if and only if 
the relation associated with G (see Section 9.3) is sym- 
metric. 


Show that if a bipartite graph G = (V, E) is n-regular for 
some positive integer n (see the preamble to Exercise 55) 
and (Vj, V2) is a bipartition of V, then |V,| = |V;|. That 
is, show that the two sets in a bipartition of the vertex set 
of an n-regular graph must contain the same number of 
vertices. 


Draw the mesh network for interconnecting nine parallel 
processors. 


In a variant of a mesh network for interconnecting 
n= m processors, processor P(i, j) is connected to 
the four processors P((i + 1) mod m, j) and P(i, (j + 1) 
mod m), so that connections wrap around the edges of 
the mesh. Draw this variant of the mesh network for 
16 processors. 


Show that every pair of processors in a mesh network 
of n = m? processors can communicate using O( yn) = 
O(m) hops between directly connected processors. 


10.3 Representing Graphs and Graph Isomorphism 703 


Representing Graphs and Graph Isomorphism 


EXAMPLE 1 


EXAMPLE 2 


10.3.1 Introduction 


There are many useful ways to represent graphs. As we will see throughout this chapter, in 
working with a graph it is helpful to be able to choose its most convenient representation. In 
this section we will show how to represent graphs in several different ways. 

Sometimes, two graphs have exactly the same form, in the sense that there is a one-to-one 
correspondence between their vertex sets that preserves edges. In such a case, we say that the 
two graphs are isomorphic. Determining whether two graphs are isomorphic is an important 
problem of graph theory that we will study in this section. 


10.3.2 Representing Graphs 
One way to represent a graph without multiple edges is to list all the edges of this graph. Another 


way to represent a graph with no multiple edges is to use adjacency lists, which specify the 
vertices that are adjacent to each vertex of the graph. 


Use adjacency lists to describe the simple graph given in Figure 1. 


Solution: Table 1 lists those vertices adjacent to each of the vertices of the graph. < 


a 
a 
> 
a 
roy 


FIGURE 1 A simple graph. 


Represent the directed graph shown in Figure 2 by listing all the vertices that are the terminal 
vertices of edges starting at each vertex of the graph. 


Solution: Table 2 represents the directed graph shown in Figure 2. < 


b TABLE 2 An Adjacency List for a 
Directed Graph. 


a 
9 


e d 


FIGURE 2 A directed graph. 
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FIGURE 3 
A simple graph. 


EXAMPLE 4 


d c 


FIGURE 4 

A graph with the 
given adjacency 
matrix. 


10.3.3 Adjacency Matrices 


Carrying out graph algorithms using the representation of graphs by lists of edges, or by adja- 
cency lists, can be cumbersome if there are many edges in the graph. To simplify computation, 
graphs can be represented using matrices. Two types of matrices commonly used to represent 
graphs will be presented here. One is based on the adjacency of vertices, and the other is based 
on incidence of vertices and edges. 

Suppose that G = (V, E) is a simple graph where |V| = n. Suppose that the vertices of G 
are listed arbitrarily as v,, v2, ..., v,. The adjacency matrix A (or Ag) of G, with respect to this 
listing of the vertices, is the n x n zero—one matrix with 1 as its (i, j)th entry when v; and y are 
adjacent, and 0 as its (i, j)th entry when they are not adjacent. In other words, if its adjacency 
matrix is A = [a;], then 


1 if {v,, v;} is an edge of G, 
d; = 
d 0 otherwise. 


Use an adjacency matrix to represent the graph shown in Figure 3. 


Solution: We order the vertices as a, b, c, d. The matrix representing this graph is 


O 1 1 1 

1 0 1 0 

1 1 0 OF 

1 0 0 0 < 
Draw a graph with the adjacency matrix 

O 1 1 0 

1 0 0 1 

1 0 0 1 

0 1 1 0 
with respect to the ordering of vertices a, b, c, d. 
Solution: A graph with this adjacency matrix is shown in Figure 4. < 


Note that an adjacency matrix of a graph is based on the ordering chosen for the vertices. 
Hence, there may be as many as n! different adjacency matrices for a graph with n vertices, 
because there are n! different orderings of n vertices. 

The adjacency matrix of a simple graph is symmetric, that is, a; = a;;, because both of 
these entries are 1 when vy; and v; are adjacent, and both are 0 otherwise. Furthermore, because 
a simple graph has no loops, each entry a; i = 1, 2,3, ..., n, is 0. 

Adjacency matrices can also be used to represent undirected graphs with loops and with 
multiple edges. A loop at the vertex v, is represented by a 1 at the (i, i)th position of the adja- 
cency matrix. When multiple edges connecting the same pair of vertices v; and v,, or multiple 
loops at the same vertex, are present, the adjacency matrix is no longer a zero—one matrix, be- 
cause the (7, j)th entry of this matrix equals the number of edges that are associated to {v, v;}. 
All undirected graphs, including multigraphs and pseudographs, have symmetric adjacency 
matrices. 


EXAMPLE 5 


a b 


d c 


FIGURE 5 
A pseudograph. 
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Use an adjacency matrix to represent the pseudograph shown in Figure 5. 


Solution: The adjacency matrix using the ordering of vertices a, b, c, d is 


ere OW 


0 
1 
1 
2 


NOW SO 
ONeK WY 


< 


We used zero—one matrices in Chapter 9 to represent directed graphs. The matrix for a 
directed graph G = (V, E) has a 1 in its (i, /)th position if there is an edge from v; to v, where 
V1, Vo, -> V, is an arbitrary listing of the vertices of the directed graph. In other words, if A = 


[a;] is the adjacency matrix for the directed graph with respect to this listing of the vertices, then 
1 ifv, vi) is an edge of G, 
Gea 

: 0 otherwise. 


The adjacency matrix for a directed graph does not have to be symmetric, because there may 
not be an edge from v; to v; when there is an edge from v; to v;. 

Adjacency matrices can also be used to represent directed multigraphs. Again, such matri- 
ces are not zero—one matrices when there are multiple edges in the same direction connecting 
two vertices. In the adjacency matrix for a directed multigraph, a,; equals the number of edges 
that are associated to (v; v;). 

TRADE-OFFS BETWEEN ADJACENCY LISTS AND ADJACENCY MATRICES When a 
simple graph contains relatively few edges, that is, when it is sparse, it is usually preferable to 
use adjacency lists rather than an adjacency matrix to represent the graph. For example, if each 
vertex has degree not exceeding c, where c is a constant much smaller than n, then each adja- 
cency list contains c or fewer vertices. Hence, there are no more than cn items in all these adja- 
cency lists. On the other hand, the adjacency matrix for the graph has n? entries. Note, however, 
that the adjacency matrix of a sparse graph is a sparse matrix, that is, a matrix with few nonzero 
entries, and there are special techniques for representing, and computing with, sparse matrices. 

Now suppose that a simple graph is dense, that is, suppose that it contains many edges, such 
as a graph that contains more than half of all possible edges. In this case, using an adjacency 
matrix to represent the graph is usually preferable over using adjacency lists. To see why, we 
compare the complexity of determining whether the possible edge {v;, v;} is present. Using an 
adjacency matrix, we can determine whether this edge is present by examining the (i, j)th entry 
in the matrix. This entry is 1 if the graph contains this edge and is 0 otherwise. Consequently, 
we need make only one comparison, namely, comparing this entry with 0, to determine whether 
this edge is present. On the other hand, when we use adjacency lists to represent the graph, we 
need to search the list of vertices adjacent to either v; or v; to determine whether this edge is 
present. This can require ©(|V|) comparisons when many edges are present. 


10.3.4 Incidence Matrices 


Another common way to represent graphs is to use incidence matrices. Let G = (V, E) be an 
undirected graph. Suppose that v,,v,...,v, are the vertices and e}, ez, ..., €„ are the edges 
of G. Then the incidence matrix with respect to this ordering of V and E is the n X m matrix 
M = [m,], where 


i 


1 when edge e; is incident with v, 
m.. = 
O otherwise. 
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EXAMPLE 6 

vy v ez v3 
e3 
ey 25 
ey 

V4 V5 
FIGURE6 An 
undirected graph. 

EXAMPLE 7 


FIGURE 7 
A pseudograph. 


Definition 1 


Represent the graph shown in Figure 6 with an incidence matrix. 


Solution: The incidence matrix is 


e ĉe ez ey C5 & 
vy}1 1 0 0 0 0 
»wv |0 O 1 1 O0 1 
v» |0 0 0 0 1 1 
wil 0 1 0 0O 0 
|o 1 0 1 1 0 d 


Incidence matrices can also be used to represent multiple edges and loops. Multiple edges 
are represented in the incidence matrix using columns with identical entries, because these edges 
are incident with the same pair of vertices. Loops are represented using a column with exactly 
one entry equal to 1, corresponding to the vertex that is incident with this loop. 


Represent the pseudograph shown in Figure 7 using an incidence matrix. 


Solution: The incidence matrix for this graph is 


e; ĉe €3 C4 C5 C6 €7 eg 


v, fl 1 10 0 0 0 0 
wok 1 10110 
»]|o 0 0 1 1000 
wld 0 0 0 0 0O 1 4 
xlo 0 0 0 1 100 


10.3.5 Isomorphism of Graphs 


We often need to know whether it is possible to draw two graphs in the same way. That is, do 
the graphs have the same structure when we ignore the identities of their vertices? For instance, 
in chemistry, graphs are used to model chemical compounds (in a way we will describe later). 
Different compounds can have the same molecular formula but can differ in structure. Such 
compounds can be represented by graphs that cannot be drawn in the same way. The graphs 
representing previously known compounds can be used to determine whether a supposedly new 
compound has been studied before. 
There is a useful terminology for graphs with the same structure. 


The simple graphs G, = (V,, E,) and G, = (V,, E,) are isomorphic if there exists a one-to- 
one and onto function f from V, to V, with the property that a and b are adjacent in G, if 
and only if f(a) and f(b) are adjacent in G5, for all a and b in V,. Such a function f is called 
an isomorphism.* Two simple graphs that are not isomorphic are called nonisomorphic. 


In other words, when two simple graphs are isomorphic, there is a one-to-one correspondence 
between vertices of the two graphs that preserves the adjacency relationship. Isomorphism of 
simple graphs is an equivalence relation. (We leave the verification of this as Exercise 49.) 


*The word isomorphism comes from the Greek roots isos for “equal” and morphe for “form.” 


EXAMPLE 8 


V3 v4 
H 


FIGURE 8 The 
graphs G and H. 
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Extra » 
Examples 


EXAMPLE 10 
b 
a Cc 
e d 
G 
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Show that the graphs G = (V, E) and H = (W, F), displayed in Figure 8, are isomorphic. 


Solution: The function f with f(u,) = v1, f(u2) = v4, f (u3) = v3, and f(u4) = v3 is a one-to-one 
correspondence between V and W. To see that this correspondence preserves adjacency, note 
that adjacent vertices in G are u and u,, u; and u3, u and u4, and u, and u4, and each of the pairs 
fu) = v; and f(u) = v4, f(u) = v; and f(uz) = v3, f (u2) = v4 and f (u4) = v2, and f(u3) = v3 
and f (u4) = v, consists of two adjacent vertices in H. < 


10.3.6 Determining whether Two Simple Graphs are Isomorphic 


It is often difficult to determine whether two simple graphs are isomorphic. There are n! possi- 
ble one-to-one correspondences between the vertex sets of two simple graphs with n vertices. 
Testing each such correspondence to see whether it preserves adjacency and nonadjacency is 
impractical if n is at all large. 

Sometimes it is not hard to show that two graphs are not isomorphic. In particular, we can 
show that two graphs are not isomorphic if we can find a property only one of the two graphs 
has, but that is preserved by isomorphism. A property preserved by isomorphism of graphs is 
called a graph invariant. For instance, isomorphic simple graphs must have the same number 
of vertices, because there is a one-to-one correspondence between the sets of vertices of the 
graphs. 

Isomorphic simple graphs also must have the same number of edges, because the one-to- 
one correspondence between vertices establishes a one-to-one correspondence between edges. 
In addition, the degrees of the vertices in isomorphic simple graphs must be the same. That is, 
a vertex v of degree d in G must correspond to a vertex f(v) of degree d in H, because a vertex 
w in G is adjacent to v if and only if f(v) and f(w) are adjacent in H. 


Show that the graphs displayed in Figure 9 are not isomorphic. 


Solution: Both G and H have five vertices and six edges. However, H has a vertex of degree one, 
namely, e, whereas G has no vertices of degree one. It follows that G and H are not isomorphic. 


< 


The number of vertices, the number of edges, and the number of vertices of each degree 
are all invariants under isomorphism. If any of these quantities differ in two simple graphs, 
these graphs cannot be isomorphic. However, when these invariants are the same, it does not 
necessarily mean that the two graphs are isomorphic. There are no useful sets of invariants 
currently known that can be used to determine whether simple graphs are isomorphic. 


Determine whether the graphs shown in Figure 10 are isomorphic. 


H G H 


FIGURE 9 The graphs G and H. FIGURE 10 The graphs G and H. 
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v 


FIGURE 11 The 
subgraphs of G 
and H made up 

of vertices of 
degree three and 
the edges con- 
necting them. 


EXAMPLE 11 


Solution: The graphs G and H both have eight vertices and 10 edges. They also both have four 
vertices of degree two and four of degree three. Because these invariants all agree, it is still 
conceivable that these graphs are isomorphic. 

However, G and H are not isomorphic. To see this, note that because deg(a) = 2 in G, a 
must correspond to either t, u, x, or y in H, because these are the vertices of degree two in H. 
However, each of these four vertices in H is adjacent to another vertex of degree two in H, which 
is not true for a in G. 

Another way to see that G and H are not isomorphic is to note that the subgraphs of G and H 
made up of vertices of degree three and the edges connecting them must be isomorphic if these 
two graphs are isomorphic (the reader should verify this). However, these subgraphs, shown in 
Figure 11, are not isomorphic. < 


To show that a function f from the vertex set of a graph G to the vertex set of a graph H is an 
isomorphism, we need to show that f preserves the presence and absence of edges. One helpful 
way to do this is to use adjacency matrices. In particular, to show that f is an isomorphism, we 
can show that the adjacency matrix of G is the same as the adjacency matrix of H, when rows 
and columns are labeled to correspond to the images under f of the vertices in G that are the 
labels of these rows and columns in the adjacency matrix of G. We illustrate how this is done 
in Example 11. 


Determine whether the graphs G and H displayed in Figure 12 are isomorphic. 


Solution: Both G and H have six vertices and seven edges. Both have four vertices of degree two 
and two vertices of degree three. It is also easy to see that the subgraphs of G and H consisting 
of all vertices of degree two and the edges connecting them are isomorphic (as the reader should 
verify). Because G and H agree with respect to these invariants, it is reasonable to try to find an 
isomorphism f. 


uy Uy vi V3 


Us 


U4 Uz V5 V4 


FIGURE 12 Graphs G and H. 


We now will define a function f and then determine whether it is an isomorphism. Because 
deg(u,) = 2 and because u, is not adjacent to any other vertex of degree two, the image of u 
must be either v, or vg, the only vertices of degree two in H not adjacent to a vertex of degree 
two. We arbitrarily set f(w,) = vę. [If we found that this choice did not lead to isomorphism, we 
would then try f(u,) = v4.] Because u, is adjacent to u,, the possible images of u, are v3 and vs. 
We arbitrarily set f(u) = v}. Continuing in this way, using adjacency of vertices and degrees 
as a guide, we set f(u3) = v4, f(u4) = v5, f(us) = vı, and f(u¢) = v. We now have a one-to- 
one correspondence between the vertex set of G and the vertex set of H, namely, f(u,) = ve» 
fan) = v3, f (u3) = V4, f(g) = V5. f(s) = v1, (Ug) = vz. To see whether f preserves edges, we 
examine the adjacency matrix of G, 


Links » 
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and the adjacency matrix of H with the rows and columns labeled by the images of the corre- 
sponding vertices in G, 


v%}0 1 0 1 0 0 
mt O 1 0 0 1 
A=%|0 10100 
v|1 0 1 0 1 0 
»lo 0 0 1 0 1 
»ļo 1 0010 


Because Ag = Ayp, it follows that f preserves edges. We conclude that f is an isomorphism, so 
G and H are isomorphic. Note that if f turned out not to be an isomorphism, we would not have 
established that G and H are not isomorphic, because another correspondence of the vertices in 
G and H may be an isomorphism. < 


ALGORITHMS FOR GRAPH ISOMORPHISM Until recently, the best algorithms known 
for determining whether two graphs are isomorphic have exponential worst-case time complex- 
ity (in the number of vertices of the graphs). However, in early 2017 László Babai announced 
that he had an algorithm that determines whether two graphs with n vertices are isomorphic 
in 2/ time, where f(n) is O(log n)*). This big-O estimate means that the algorithm runs in 
quasi-polynomial time, which is between polynomial time and exponential time. This discov- 
ery of Babai has not yet been fully peer reviewed, but he was able to correct a serious gap in 
an algorithm he invented in 2015. Experts believe that his result is now correct. Even though 
no polynomial-time algorithms have been found for solving this problem, it can be solved by 
linear average-case time complexity algorithms. There is some hope, but also skepticism, that 
an algorithm with polynomial worst-case time complexity for determining whether two graphs 
are isomorphic can be found. 

The best practical general purpose software for isomorphism testing, called NAUTY, can 
be used to determine whether two graphs with as many as 100 vertices are isomorphic in less 
than a second on a modern PC. NAUTY software can be downloaded over the Internet and 
experimented with. Practical algorithms for determining whether two graphs are isomorphic 
exist for graphs that are restricted in various ways, such as when the maximum degree of the 
vertices is small. The problem of determining whether any two graphs are isomorphic is of 
special interest because it is one of only a few NP problems (see Exercise 78) not known to be 
either tractable or NP-complete (see Section 3.3). 


APPLICATIONS OF GRAPH ISOMORPHISMS Graph isomorphisms, and functions that 
are almost graph isomorphisms, arise in applications of graph theory to chemistry and to the 
design of electronic circuits, and other areas including bioinformatics and computer vision. 
Chemists use multigraphs, known as molecular graphs, to model chemical compounds. In these 
graphs, vertices represent atoms and edges represent chemical bonds between these atoms. Two 
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structural isomers, molecules with identical molecular formulas but with atoms bonded differ- 
ently, have nonisomorphic molecular graphs. When a potentially new chemical compound is 
synthesized, a database of molecular graphs is checked to see whether the molecular graph of 
the compound is the same as one already known. 

Electronic circuits are modeled using graphs in which vertices represent components and 
edges represent connections between them. Modern integrated circuits, known as chips, are 
miniaturized electronic circuits, often with millions of transistors and connections between 
them. Because of the complexity of modern chips, automation tools are used to design them. 
Graph isomorphism is the basis for the verification that a particular layout of a circuit produced 
by an automated tool corresponds to the original schematic of the design. Graph isomorphism 
can also be used to determine whether a chip from one vendor includes intellectual property 
from a different vendor. This can be done by looking for large isomorphic subgraphs in the 
graphs modeling these chips. 

Exercises 
In Exercises 1—4 use an adjacency list to represent the given 12./; 1 1 0 
graph. 0010 
1 0 1 0 
Ll. a 2.4 b c 111 0 
In Exercises 13—15 represent the given graph using an adja- 
cency matrix. 
c ʻ e 13. a b 14. a b 
3. 
c d c d 
15. 
5. Represent the graph in Exercise 1 with an adjacency ma- a b 
trix. 
6. Represent the graph in Exercise 2 with an adjacency ma- 
trix. z d 
7. Represent the graph in Exercise 3 with an adjacency ma- 
trix. 
: . . ; In Exercises 16-18 draw an undirected graph represented by 
8. -i the graph in Exercise 4 with an adjacency ma- the given adjacency matrix. 
T1X. 
9. Represent each of these graphs with an adjacency matrix. 16.[; 3 2 lt 2 0 1 
a) K, b) Ky4 ©) Ky; 3 0 4 2 0 30 
d) C, e) W, f) Q; LAN oa. l 
1 0 1 0 
In Exercises 10-12 draw a graph with the given adjacency 
matrix. 
110 1 3 0 4 
10. jo 1 0 1. fo o 1 1 1 2 1 30 
1 0 1 0 0 1 0 3 1 1 0 1 
0 1 0 1 1 0 1 0 3 0 0 2 
1 1 1 0 4 0 1 2 3 


In Exercises 19-21 find the adjacency matrix of the given di- 
rected multigraph with respect to the vertices listed in alpha- 
betic order. 


19. 


a 


21. 


In Exercises 22-24 draw the graph represented by the given 
adjacency matrix. 


22.11 0 1f 2311 2 1| fo 2 3 0 
0 0 1 2 0 0 I 2 2 1 

I i 1 02 2 2 1h. 

1 0 0 2 


The density of an undirected graph G is the number of edges 
of G divided by the number of possible edges in an undi- 
rected graph with |G| vertices. Consequently, the density of 
G(V, E) is 


2|E| 
IVIdVI- D° 


A family of graphs G,,, n = 1, 2, ... is sparse if the limit of 

the density of G,, is zero as n grows without bound, while it 

is dense if this proportion approaches a positive real number. 

As mentioned in the text, an individual graph is called sparse 

when it contains relatively few edges and dense if it contains 

many edges. These terms can be defined precisely depend- 
ing on the context, but different definitions generally will not 
agree. 

25. Find the density of the graph in 
a) Figure 1 of Section 10.1. 

b) Figure 6 of Section 10.1. 
c) Figure 12 of Section 10.1. 

26. Find the density of the graph in 
a) Figure 12 of Section 10.2. 

b) Figure 13 of Section 10.2. 
c) Figure 14 of Section 10.2. 

27. (Requires calculus) For each of these families of graphs, 
determine whether the graph family is sparse, dense, or 
neither. (Refer to the results of Exercise 37 in Section 10.2.) 
a) K,, b) Cn c) Kin 
d) Qn e) W, f) K3 n 

28. Determine whether an undirected graph is sparse, dense, 
or neither, and explain your answer, if it is used to 
model 


29. 


30. 


31. 


*32. 


*33. 


34. 


35. 


* 36. 


* 37. 
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a) the street network in a city (where the vertices are 
street intersections). 


b) whether buildings in a city are within two miles of one 
another. 


c) whether two people in the world are siblings. 


d) whether two people work for the same company. 

Is every zero—one square matrix that is symmetric and 
has zeros on the diagonal the adjacency matrix of a sim- 
ple graph? 

Use an incidence matrix to represent the graphs in Exer- 
cises 1 and 2. 


Use an incidence matrix to represent the graphs in Exer- 
cises 13-15. 


What is the sum of the entries in a row of the adjacency 
matrix for an undirected graph? For a directed graph? 


What is the sum of the entries in a column of the ad- 
jacency matrix for an undirected graph? For a directed 
graph? 

What is the sum of the entries in a row of the incidence 
matrix for an undirected graph? 


What is the sum of the entries in a column of the inci- 
dence matrix for an undirected graph? 


Find an adjacency matrix for each of these graphs. 
a) K,, b) Cn c) W, d) K mn e) Qn 


Find incidence matrices for the graphs in parts (a)—(d) of 
Exercise 36. 


In Exercises 38-48 determine whether the given pair of 
graphs is isomorphic. Exhibit an isomorphism or provide a 
rigorous argument that none exists. For additional exercises 
of this kind, see Exercises 3-5 in the Supplementary Exer- 
cises. 


38. vy v 
u Uy U3 ug Us v3 
e & d d @ 
V4 V5 
39. v2 
Uy 
vy V3 
uy u3 
Us uy V5 V4 
40. u u vi 
V5 V9 
uy U3 v4 v3 
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v 
41. u u 1 
v7 
uz U3 
V6 
ug ug 
Us vs V4 
42. u Uy vi 
v2 
V5 
Us Ug U3 V4 
43. uy 
Yi 1) 
v4 v3 
U4 
44. vi Vo 
uy Uy 
X> ú me 
Us U4 V5 v4 
45. uy, Uy U3 Us U6 Ug 
e © e ® 
ug Uy 
vy v2 V4 V5 V6 Vg 
o 


<x 
wo 

< 
x 


V3 


46. 


47. 


48. 


49. 


50. 


51. 


52. 


53. 


ug u7 
uy Uy U3 U4 Us 
° © 

Ug Ug uio 

V6 V7 
vi v V3 V4 V5 
e D 

Vg Vo Vio 

uz v2 


Us U4 vs 
uy vy 
ug un V8 v2 
uz u3 v7 V3 
U6 U4 V6 V4 
Us V5 


Show that isomorphism of simple graphs is an equiva- 
lence relation. 


Suppose that G and H are isomorphic simple graphs. 


Show that their complementary graphs G and H are also 
isomorphic. 


Describe the row and column of an adjacency matrix of 
a graph corresponding to an isolated vertex. 


Describe the row of an incidence matrix of a graph cor- 
responding to an isolated vertex. 


Show that the vertices of a bipartite graph with two or 
more vertices can be ordered so that its adjacency matrix 
has the form 


so} 


where the four entries shown are rectangular blocks. 


A simple graph G is called self-complementary if G and G 
are isomorphic. 


54. 


55. 


* 56. 


57. 
58. 


59. 


60. 


*61. 


62. 


63. 


64. 


65. 


66. 


Show that this graph is self-complementary. 


a b 


d c 
Find a self-complementary simple graph with five ver- 


tices. 


Show that if G is a self-complementary simple graph 
with v vertices, then v = 0 or 1 (mod 4). 


For which integers n is C,, self-complementary? 


How many nonisomorphic simple graphs are there with n 
vertices, when n is 

a) 2? b) 3? c) 4? 

How many nonisomorphic simple graphs are there with 
five vertices and three edges? 


How many nonisomorphic simple graphs are there with 
six vertices and four edges? 


Find the number of nonisomorphic simple graphs with 
six vertices in which each vertex has degree three. 


Find the number of nonisomorphic simple graphs with 
seven vertices in which each vertex has degree two. 


Are the simple graphs with the following adjacency ma- 
trices isomorphic? 


a){O 0 1//0 1 1 
0 0 Ihi 0 Ô 
1 1 Of]1 0 0 
b)joO 1 O 1/J/0 1 1 1 
1 0 0 HII 00 1 
0 0 0 I1f]}]1 0 0 1 
1 1 1 Of f1 1 ið 
c)}0 1 1 oO; JO 1 0 +1 
1 0 0 If]1 0 0 0 
1 0 0 I1f]0 0 0 1 
0 1 1 O}f1 0 1 0 


Determine whether the graphs without loops with these 
incidence matrices are isomorphic. 


a)/1 0O Il} ]1 1 0 
Oo 1 hL 0 1 
1 1 Of JO 1 1 

b)}/1 1 0 0 O] JO 1 00 1 
101 0 Ilio 1 1 21 0 
00 0 1 I1;}1 0 0 1 o0 
Oo 1 1 1 Of}]1 0 1 0 1 


Extend the definition of isomorphism of simple graphs to 
undirected graphs containing loops and multiple edges. 


Define isomorphism of directed graphs. 
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In Exercises 67-70 determine whether the given pair of di- 
rected graphs are isomorphic. (See Exercise 66.) 


67. 


68. 


69. 


70. 


71. 


72. 


73. 


*74, 


*75. 


* 76, 


uy Uy vy V9 
ux ug V3 V4 
uy Uy vy V9 
U3 U4 V3 V4 
uy vy 
Uy U3 v2 V3 
uy Uy U3 
U4 Us U6 
vy v 
<> 
V5 v4 


Show that if G and H are isomorphic directed graphs, 
then the converses of G and H (defined in the preamble 
of Exercise 69 of Section 10.2) are also isomorphic. 
Show that the property that a graph is bipartite is an iso- 
morphic invariant. 

Find a pair of nonisomorphic graphs with the same de- 
gree sequence (defined in the preamble to Exercise 38 
in Section 10.2) such that one graph is bipartite, but the 
other graph is not bipartite. 

How many nonisomorphic directed simple graphs are 
there with n vertices, when n is 

a) 2? b) 3? c) 4? 

What is the product of the incidence matrix and its trans- 
pose for an undirected graph? 

How much storage is needed to represent a simple graph 
with n vertices and m edges using 

a) adjacency lists? 


b) an adjacency matrix? 
c) an incidence matrix? 
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A devil’s pair for a purported isomorphism test is a pair of 78. Suppose that the function f from V, to V, is an 

nonisomorphic graphs that the test fails to show that they are isomorphism of the graphs G, = (V,,£,) and G, = 

not isomorphic. (V,, E2). Show that it is possible to verify this fact in 

77. Find a devil’s pair for the test that checks the degree se- time polynomial in terms of the number of vertices 
quence (defined in the preamble to Exercise 38 in Sec- of the graph, in terms of the number of comparisons 
tion 10.2) in two graphs to make sure they agree. needed. 


Connectivity 


Definition 1 


10.4.1 Introduction 


Many problems can be modeled with paths formed by traveling along the edges of graphs. For 
instance, the problem of determining whether a message can be sent between two computers 
using intermediate links can be studied with a graph model. Problems of efficiently planning 
routes for mail delivery, garbage pickup, diagnostics in computer networks, and so on can be 
solved using models that involve paths in graphs. 


10.4.2 Paths 


Informally, a path is a sequence of edges that begins at a vertex of a graph and travels from vertex 
to vertex along edges of the graph. As the path travels along its edges, it visits the vertices along 
this path, that is, the endpoints of these edges. 

A formal definition of paths and related terminology is given in Definition 1. 


Let n be a nonnegative integer and G an undirected graph. A path of length n from u 
to v in G is a sequence of n edges e,,...,e, of G for which there exists a sequence 
Xp =U, Xy, ..-,Xp_1,X, = V Of vertices such that e,; has, for i= 1,...,n, the endpoints x;_, 
and x;. When the graph is simple, we denote this path by its vertex sequence Xo, X1, ...,X, 
(because listing these vertices uniquely determines the path). The path is a circuit if it begins 
and ends at the same vertex, that is, if u = v, and has length greater than zero. The path or 
circuit is said to pass through the vertices x,,X, ...,X,_, Or traverse the edges e}, e2, ..., € 
A path or circuit is simple if it does not contain the same edge more than once. 


n* 


When it is not necessary to distinguish between multiple edges, we will denote a path 
€1,€9,...,@,, where e; is associated with {x,;_,,x;} for i= 1,2, ...,n by its vertex sequence 
Xo X1,-..,X,. This notation identifies a path only as far as which vertices it passes through. 
Consequently, it does not specify a unique path when there is more than one path that passes 
through this sequence of vertices, which will happen if and only if there are multiple edges be- 
tween some successive vertices in the list. Note that a path of length zero consists of a single 
vertex. 


Remark: There is considerable variation of terminology concerning the concepts defined 
in Definition 1. For instance, in some books, the term walk is used instead of path, 
where a walk is defined to be an alternating sequence of vertices and edges of a graph, 
Vo Ep Vp Cas «++» Vat» Ew Vy» Where v;_; and v; are the endpoints of e; for i = 1, 2, ...,n. When 
this terminology is used, closed walk is used instead of circuit to indicate a walk that begins and 
ends at the same vertex, and trail is used to denote a walk that has no repeated edge (replacing 
the term simple path). When this terminology is used, the terminology path is often used for 
a trail with no repeated vertices, conflicting with the terminology in Definition 1. Because of 
this variation in terminology, you will need to make sure which set of definitions are used in a 
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FIGURE 1 A simple graph. 


particular book or article when you read about traversing edges of a graph. The text [GrYe06] 
is a good reference for the alternative terminology described in this remark. 


In the simple graph shown in Figure 1, a, d, c, f, e is a simple path of length 4, because {a, d}, 
{d, c}, {cf}, and { f, e} are all edges. However, d, e, c, a is not a path, because {e, c} is not an 
edge. Note that b, c, f, e, b is a circuit of length 4 because {b, c}, {c, f }, {fe}, and {e, b} are 
edges, and this path begins and ends at b. The path a, b, e, d, a, b, which is of length 5, is not 
simple because it contains the edge {a, b} twice. < 


Paths and circuits in directed graphs were introduced in Chapter 9. We now provide more 
general definitions. 


Let n be a nonnegative integer and G a directed graph. A path of length n from u to v in G is 
a sequence of edges e}, e2, ..., €„ Of G such that e, is associated with (xo, x), e3 is associated 
with (x, X2), and so on, with e, associated with (x„—1 X„,), where x) = u and x, = v. When 
there are no multiple edges in the directed graph, this path is denoted by its vertex sequence 
Xo Xp X2, ...,X,. A path of length greater than zero that begins and ends at the same vertex 
is called a circuit or cycle. A path or circuit is called simple if it does not contain the same 
edge more than once. 


Remark: Terminology other than that given in Definition 2 is often used for the concepts de- 
fined there. In particular, the alternative terminology that uses walk, closed walk, trail, and path 
(described in the remarks following Definition 1) may be used for directed graphs. See [GrYe05] 
for details. 


Note that the terminal vertex of an edge in a path is the initial vertex of the next edge in the 
path. When it is not necessary to distinguish between multiple edges, we will denote a path 
ejs €% -> €p Where e; is associated with (x,_,,x;) for i= 1,2, ..., n, by its vertex sequence 
Xo Xp -..,X,. The notation identifies a path only as far as which the vertices it passes through. 
There may be more than one path that passes through this sequence of vertices, which will 
happen if and only if there are multiple edges between two successive vertices in the list. 


Paths represent useful information in many graph models, as Examples 2—4 demonstrate. 


Paths in Acquaintanceship Graphs In an acquaintanceship graph there is a path between 
two people if there is a chain of people linking these people, where two people adjacent in 
the chain know one another. For example, in Figure 6 in Section 10.1, there is a chain of six 
people linking Kamini and Ching. Many social scientists have conjectured that almost every 
pair of people in the world are linked by a small chain of people, perhaps containing just five or 
fewer people. This would mean that almost every pair of vertices in the acquaintanceship graph 
containing all people in the world is linked by a path of length not exceeding four. The play Six 
Degrees of Separation by John Guare is based on this notion. < 
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TABLE 1 The Number TABLE 2 The Number 
of Mathematicians of Actors with a Given 
with a Given Erdős Bacon Number (as of 
Number (as of early August 2017). 


2006). 
Bacon Number 
Erdős Number Number of People 
of People 


Number 
1 
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4,388 
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9 
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Paths in Collaboration Graphs In a collaboration graph, two people a and b are connected 
by a path when there is a sequence of people starting with a and ending with b such that the 
endpoints of each edge in the path are people who have collaborated. We will consider two 
particular collaboration graphs here. First, in the academic collaboration graph of people who 
have written papers in mathematics, the Erdős number of a person m (defined in terms of 
relations in Supplementary Exercise 14 in Chapter 9) is the length of the shortest path between 
m and the extremely prolific mathematician Paul Erdős (who died in 1996). That is, the Erdős 
number of a mathematician is the length of the shortest chain of mathematicians that begins with 
Paul Erdős and ends with this mathematician, where each adjacent pair of mathematicians have 
written a joint paper. The number of mathematicians with each Erdős number, last computed in 
2006 as part of the Erdős Number Project, is shown in Table 1. 

In the Hollywood graph (see Example 3 in Section 10.1), two actors a and b are linked when 
there is a chain of actors linking a and b, where every two actors adjacent in the chain have acted 
in the same movie. In the Hollywood graph, the Bacon number of an actor c is defined to be the 
length of the shortest path connecting c and the well-known actor Kevin Bacon. As new movies 
are made, including new ones with Kevin Bacon, the Bacon number of actors can change. In 
Table 2 we show the number of actors with each Bacon number as of August 2017 using data 
from the Oracle of Bacon website. The origins of the Bacon number of an actor dates back to the 
early 1990s, when Kevin Bacon remarked that he had worked with everyone in Hollywood or 
someone who worked with them. This led some people to invent a party game where participants 
where challenged to find a sequence of movies leading from each actor named to Kevin Bacon. 
Before the web was born, you needed to be a movie expert to play this game well. Now, you 
need only consult the Oracle of Bacon. We can find a number similar to a Bacon number using 
any actor as the center of the acting universe. < 


10.4.3 Connectedness in Undirected Graphs 


When does a computer network have the property that every pair of computers can share infor- 
mation, if messages can be sent through one or more intermediate computers? When a graph 
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is used to represent this computer network, where vertices represent the computers and edges 
represent the communication links, this question becomes: When is there always a path between 
two vertices in the graph? 


An undirected graph is called connected if there is a path between every pair of distinct 
vertices of the graph. An undirected graph that is not connected is called disconnected. We 
say that we disconnect a graph when we remove vertices or edges, or both, to produce a 
disconnected subgraph. 


Thus, any two computers in the network can communicate if and only if the graph of this network 
is connected. 


The graph G, in Figure 2 is connected, because for every pair of distinct vertices there is a 
path between them (the reader should verify this). However, the graph G, in Figure 2 is not 
connected. For instance, there is no path in G, between vertices a and d. < 


We will need the following theorem in Chapter 11. 


There is a simple path between every pair of distinct vertices of a connected undirected graph. 


Proof: Let u and v be two distinct vertices of the connected undirected graph G = (V, E). Be- 
cause G is connected, there is at least one path between u and v. Let x9, X1, ..., Xp, where x) = u 
and x,, = v, be the vertex sequence of a path of least length. This path of least length is simple. 
To see this, suppose it is not simple. Then x; = x for some i and j with 0 <i < j. This means 
that there is a path from u to v of shorter length with vertex sequence Xp, x), ...,X;1, Ny soon X 


obtained by deleting the edges corresponding to the vertex sequence x, ..., X1- 


n 


CONNECTED COMPONENTS A connected component of a graph G is a connected sub- 
graph of G that is not a proper subgraph of another connected subgraph of G. Thatis, a connected 
component of a graph G is a maximal connected subgraph of G. A graph G that is not connected 
has two or more connected components that are disjoint and have G as their union. 


What are the connected components of the graph H shown in Figure 3? 


Solution: The graph H is the union of three disjoint connected subgraphs H,, H,, and H}, shown 
in Figure 3. These three subgraphs are the connected components of H. < 


a b 


w 
o 
a 

s 

= 


G, G» 


FIGURE 2 The graphs G, and FIGURE 3 The graph H and its 
Gz. connected components H,, H,, and H}. 
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Connected Components of Call Graphs Two vertices x and y are in the same component of a 
telephone call graph (see Example 4 in Section 10.1) when there is a sequence of telephone calls 
beginning at x and ending at y. When a call graph for telephone calls made during a particular 
day in the AT&T network was analyzed, this graph was found to have 53,767,087 vertices, more 
than 170 million edges, and more than 3.7 million connected components. Most of these com- 
ponents were small; approximately three-fourths consisted of two vertices representing pairs of 
telephone numbers that called only each other. This graph has one huge connected component 
with 44,989,297 vertices comprising more than 80% of the total. Furthermore, every vertex in 
this component can be linked to any other vertex by a chain of no more than 20 calls. < 


10.4.4 How Connected is a Graph? 


Suppose that a graph represents a computer network. Knowing that this graph is connected tells 
us that any two computers on the network can communicate. However, we would also like to 
understand how reliable this network is. For instance, will it still be possible for all computers to 
communicate after a router or a communications link fails? To answer this and similar questions, 
we now develop some new concepts. 

Sometimes the removal from a graph of a vertex and all incident edges produces a subgraph 
with more connected components. Such vertices are called cut vertices (or articulation points). 
The removal of a cut vertex from a connected graph produces a subgraph that is not connected. 
Analogously, an edge whose removal produces a graph with more connected components than 
in the original graph is called a cut edge or bridge. Note that in a graph representing a computer 
network, a cut vertex and a cut edge represent an essential router and an essential link that cannot 
fail for all computers to be able to communicate. 


Find the cut vertices and cut edges in the graph G, shown in Figure 4. 


Solution: The cut vertices of G, are b, c, and e. The removal of one of these vertices (and its 
adjacent edges) disconnects the graph. The cut edges are {a, b} and {c, e}. Removing either one 
of these edges disconnects G}. < 


VERTEX CONNECTIVITY Not all graphs have cut vertices. For example, the complete 
graph K,,, where n > 3, has no cut vertices. When you remove a vertex from K, and all edges 
incident to it, the resulting subgraph is the complete graph K,,_,, a connected graph. Connected 
graphs without cut vertices are called nonseparable graphs, and can be thought of as more con- 
nected than those with a cut vertex. We can extend this notion by defining a more granulated 
measure of graph connectivity based on the minimum number of vertices that can be removed 
to disconnect a graph. 

A subset V’ of the vertex set V of G = (V, E) is a vertex cut, or separating set, if G — V’ 
is disconnected. For instance, in the graph in Figure 1, the set {b, c, e} is a vertex cut with three 
vertices, as the reader should verify. We leave it to the reader (Exercise 51) to show that every 
connected graph, except a complete graph, has a vertex cut. We define the vertex connectivity 
of a noncomplete graph G, denoted by x(G), as the minimum number of vertices in a vertex cut. 

When G is a complete graph, it has no vertex cuts, because removing any subset of its 
vertices and all incident edges still leaves a complete graph. Consequently, we cannot de- 
fine x(G) as the minimum number of vertices in a vertex cut when G is complete. Instead, 
we set «(K,,) =n — 1, the number of vertices needed to be removed to produce a graph with a 
single vertex. 

Consequently, for every graph G, x(G) is minimum number of vertices that can be re- 
moved from G to either disconnect G or produce a graph with a single vertex. We have 


EXAMPLE 8 


10.4 Connectivity 719 


a 
a 

n 

o 
a 
o 
a 


> 
S 
a 
> 
Sey 
a 


Gi G, 


5 
a 
> 
o 

a 


Gs 


FIGURE 4 Some connected graphs. 


0 < «(G) <n-1 if G has n vertices, x(G) = 0 if and only if G is disconnected or G = K}, 
and x(G) = n — 1 if and only if Gis complete [see Exercise 52(a)]. 

The larger x(G) is, the more connected we consider G to be. Disconnected graphs and K, 
have x(G) = 0, connected graphs with cut vertices and K, have x(G) = 1, graphs without cut 
vertices that can be disconnected by removing two vertices and K, have x(G) = 2, and so 
on. We say that a graph is k-connected (or k-vertex-connected), if x(G) > k. A graph G is 
1-connected if it is connected and not a graph containing a single vertex; a graph is 2-connected, 
or biconnected, if it is nonseparable and has at least three vertices. Note that if G is a k- 
connected graph, then G is a j-connected graph for all j with O < j < k. 


Find the vertex connectivity for each of the graphs in Figure 4. 


Solution: Each of the five graphs in Figure 4 is connected and has more than vertex, so each 
of these graphs has positive vertex connectivity. Because G, is a connected graph with a cut 
vertex, as shown in Example 7, we know that «(G,) = 1. Similarly, x(G) = 1, because c is a 
cut vertex of Gp. 

The reader should verify that G, has no cut vertices, but that {b, g} is a vertex cut. Hence, 
K(G3) = 2. Similarly, because G, has a vertex cut of size two, {c,f}, but no cut vertices. It 
follows that «(G,) = 2. The reader can verify that G; has no vertex cut of size two, but {b, c, f } 
is a vertex cut of G;. Hence, k(G;) = 3. < 
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EDGE CONNECTIVITY We can also measure the connectivity of a connected graph G = 
(V, E) in terms of the minimum number of edges that we can remove to disconnect it. If a graph 
has a cut edge, then we need only remove it to disconnect G. If G does not have a cut edge, we 
look for the smallest set of edges that can be removed to disconnect it. A set of edges E’ is called 
an edge cut of G if the subgraph G — F’ is disconnected. The edge connectivity of a graph G, 
denoted by A(G), is the minimum number of edges in an edge cut of G. This defines A(G) for 
all connected graphs with more than one vertex because it is always possible to disconnect 
such a graph by removing all edges incident to one of its vertices. Note that A(G) = 0 if G is 
not connected. We also specify that A(G) = 0 if G is a graph consisting of a single vertex. It 
follows that if G is a graph with n vertices, then 0 < A(G) < n — 1. We leave it to the reader 
[Exercise 52(b)] to show that A(G) = n — 1 where G is a graph with n vertices if and only if 
G = K,, which is equivalent to the statement that A(G) < n — 2 when Gis not a complete graph. 


n? 


Find the edge connectivity of each of the graphs in Figure 4. 


Solution: Each of the five graphs in Figure 4 is connected and has more than one vertex, so we 
know that all of them have positive edge connectivity. As we saw in Example 7, G, has a cut 
edge, so A(G,) = 1. 

The graph G, has no cut edges, as the reader should verify, but the removal of the two edges 
{a, b} and {a, c} disconnects it. Hence, A(G,) = 2. Similarly, A(G;) = 2, because G; has no cut 
edges, but the removal of the two edges {b, c} and { f, g} disconnects it. 

The reader should verify that the removal of no two edges disconnects G,, but the removal 
of the three edges {b, c}, {a, f }, and { f, g} disconnects it. Hence, A(G,) = 3. Finally, the reader 
should verify that A(G;) = 3, because the removal of any two of its edges does not disconnect 
it, but the removal of {a, b}, {a, g}, and {a, h} does. < 


AN INEQUALITY FOR VERTEX CONNECTIVITY AND EDGE CONNECTIVITY 
When G = (V, E) is a noncomplete connected graph with at least three vertices, the minimum 
degree of a vertex of G is an upper bound for both the vertex connectivity of G and the edge 
connectivity of G. That is, x(G) < min,ey deg(v) and A(G) < min,ey deg(v). To see this, ob- 
serve that deleting all the neighbors of a fixed vertex of minimum degree disconnects G, and 
deleting all the edges that have a fixed vertex of minimum degree as an endpoint disconnects G. 

In Exercise 55, we ask the reader to show that x(G) < A(G) when G is a connected non- 
complete graph. Note also that «(K,,) = A(K,,) = min,cy deg(v) = n — 1 when n is a positive 
integer and that x(G) = A(G) = 0 when G is a disconnected graph. Putting these facts together 
establishes that for all graphs G, 


K(G) < A(G) < min, cy deg(v). 


APPLICATIONS OF VERTEX AND EDGE CONNECTIVITY Graph connectivity plays an 
important role in many problems involving the reliability of networks. For instance, as we men- 
tioned in our introduction of cut vertices and cut edges, we can model a data network using 
vertices to represent routers and edges to represent links between them. The vertex connectiv- 
ity of the resulting graph equals the minimum number of routers that disconnect the network 
when they are out of service. If fewer routers are down, data transmission between every pair 
of routers is still possible. The edge connectivity represents the minimum number of fiber optic 
links that can be down to disconnect the network. If fewer links are down, it will still be possible 
for data to be transmitted between every pair of routers. 

We can model a highway network, using vertices to represent highway intersections and 
edges to represent sections of roads running between intersections. The vertex connectivity 
of the resulting graph represents the minimum number of intersections that can be closed at 
a particular time that makes it impossible to travel between every two intersections. If fewer 
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intersections are closed, travel between every pair of intersections is still possible. The edge 
connectivity represents the minimum number of roads that can be closed to disconnect the 
highway network. If fewer highways are closed, it will still be possible to travel between any 
two intersections. Clearly, it would be useful for the highway department to take this information 
into account when planning road repairs. 


10.4.5 Connectedness in Directed Graphs 


There are two notions of connectedness in directed graphs, depending on whether the directions 
of the edges are considered. 


A directed graph is strongly connected if there is a path from a to b and from b to a whenever 
a and b are vertices in the graph. 


For a directed graph to be strongly connected there must be a sequence of directed edges 
from any vertex in the graph to any other vertex. A directed graph can fail to be strongly con- 
nected but still be in “one piece.” Definition 5 makes this notion precise. 


A directed graph is weakly connected if there is a path between every two vertices in the 
underlying undirected graph. 


That is, a directed graph is weakly connected if and only if there is always a path between 
two vertices when the directions of the edges are disregarded. Clearly, any strongly connected 
directed graph is also weakly connected. 


Are the directed graphs G and H shown in Figure 5 strongly connected? Are they weakly con- 
nected? 


Solution: G is strongly connected because there is a path between any two vertices in this di- 
rected graph (the reader should verify this). Hence, G is also weakly connected. The graph H 
is not strongly connected. There is no directed path from a to b in this graph. However, H is 
weakly connected, because there is a path between any two vertices in the underlying undirected 
graph of H (the reader should verify this). < 


STRONG COMPONENTS OF A DIRECTED GRAPH The subgraphs of a directed graph G 
that are strongly connected but not contained in larger strongly connected subgraphs, that is, 
the maximal strongly connected subgraphs, are called the strongly connected components or 
strong components of G. Note that if a and b are two vertices in a directed graph, their strong 
components are either the same or disjoint. (We leave the proof of this last fact as Exercise 17.) 


G H 


FIGURE 5 The directed graphs G and H. 
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The graph H in Figure 5 has three strongly connected components, consisting of the vertex a; 
the vertex e; and the subgraph consisting of the vertices b, c, and d and edges (b, c), (c, d), and 
(d, b). 


The Strongly Connected Components of the Web Graph The web graph introduced in 
Example 5 of Section 10.1 represents web pages with vertices, and links with directed edges. 
Determining the size of the web graph is a challenging problem because it is extraordinarily 
large. A snapshot of the web in 1999 produced a web graph with over 200 million vertices and 
over 1.5 billion edges. In 2010 the web graph was estimated to have at least 55 billion vertices 
and one trillion edges. From these limited data (and from what is readily apparent if you have 
used the web in recent years), the number of web pages has a rapid rate of growth. (See [Br00] 
and the web resources for details.) 

The underlying undirected graph of the web graph, as it existed in 1999, is not connected, 
but it has a connected component that includes approximately 90% of the vertices in the graph. 
The subgraph of the original directed graph corresponding to this connected component of the 
underlying undirected graph (that is, with the same vertices and all directed edges connecting 
vertices in this subgraph) has one very large strongly connected component and many small 
ones. The former is called the giant strongly connected component (GSCC) of the directed 
graph. A web page in this component can be reached following links starting at any other page 
in this component. The GSCC in the web graph was found to have over 53 million vertices. The 
remaining vertices in the large connected component of the undirected graph represent three 
different types of web pages: pages that can be reached from a page in the GSCC, but do not 
link back to these pages following a series of links; pages that link back to pages in the GSCC 
following a series of links, but cannot be reached by following links on pages in the GSCC; 
and pages that cannot reach pages in the GSCC and cannot be reached from pages in the GSCC 
following a series of links. Each of these three other sets was found to have approximately 
44 million vertices. (It is rather surprising that these three sets are close to the same size.) <4 


10.4.6 Paths and Isomorphism 


There are several ways that paths and circuits can help determine whether two graphs are iso- 
morphic. For example, the existence of a simple circuit of a particular length is a useful invariant 
that can be used to show that two graphs are not isomorphic. In addition, paths can be used to 
construct mappings that may be isomorphisms. 

As we mentioned, a useful isomorphic invariant for simple graphs is the existence of a 
simple circuit of length k, where k is a positive integer greater than 2. (The proof that this is an 
invariant is left as Exercise 60.) Example 13 illustrates how this invariant can be used to show 
that two graphs are not isomorphic. 


Determine whether the graphs G and H shown in Figure 6 are isomorphic. 


Solution: Both Gand H have six vertices and eight edges. Each has four vertices of degree three, 
and two vertices of degree two. So, the three invariants—number of vertices, number of edges, 
and degrees of vertices—all agree for the two graphs. However, H has a simple circuit of length 
three, namely, v,, v2, Ve, v}, Whereas G has no simple circuit of length three, as can be determined 
by inspection (all simple circuits in G have length at least four). Because the existence of a simple 
circuit of length three is an isomorphic invariant, G and H are not isomorphic. < 


We have shown how the existence of a type of path, namely, a simple circuit of a particular 
length, can be used to show that two graphs are not isomorphic. We can also use paths to find 
mappings that are potential isomorphisms. 
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G H G H 
FIGURE 6 The graphs G and H. FIGURE 7 The graphs G and H. 


Determine whether the graphs G and H shown in Figure 7 are isomorphic. 


Solution: Both G and H have five vertices and six edges, both have two vertices of degree three 
and three vertices of degree two, and both have a simple circuit of length three, a simple circuit 
of length four, and a simple circuit of length five. Because all these isomorphic invariants agree, 
G and H may be isomorphic. 

To find a possible isomorphism, we can follow paths that go through all vertices so that the 
corresponding vertices in the two graphs have the same degree. For example, the paths t, tt4, 
Ux, Uy, Us in G and v3, V2, Vi, V5, v4 in H both go through every vertex in the graph; start at a 
vertex of degree three; go through vertices of degrees two, three, and two, respectively; and end 
at a vertex of degree two. By following these paths through the graphs, we define the mapping 
f with f(u,) = v3, f (u4) = Vo, f (u3) = v1, f(u) = Vs, and f (u5) = v4. The reader can show that f 
is an isomorphism, so G and H are isomorphic, either by showing that f preserves edges or by 
showing that with the appropriate orderings of vertices the adjacency matrices of G and H are 
the same. < 


10.4.7 Counting Paths Between Vertices 


The number of paths between two vertices in a graph can be determined using its adjacency 
matrix. 


Let G be a graph with adjacency matrix A with respect to the ordering v,, vz, ..., V, of the 
vertices of the graph (with directed or undirected edges, with multiple edges and loops al- 
lowed). The number of different paths of length r from v; to Vi, where r is a positive integer, 
equals the (i, j)th entry of A’. 


Proof: The theorem will be proved using mathematical induction. Let G be a graph with adja- 


cency matrix A (assuming an ordering v4, V2, ..., v, of the vertices of G). The number of paths 
from v; to v, of length 1 is the (i, j)th entry of A, because this entry is the number of edges from 
v; tov,. 

eg 


Assume that the (i, j)th entry of A” is the number of different paths of length r from v; to v;. 
This is the inductive hypothesis. Because A’*! = A’A, the (i, j)th entry of A’*! equals 


bia; + bpa; tet b, a 


in“ np 


where b; is the (i, k)th entry of A”. By the inductive hypothesis, b;, is the number of paths of 
length r from v; to v,. 
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A path of length r + 1 from v; to v, is made up of a path of length r from v; to some interme- 
diate vertex v,, and an edge from v, to v;. By the product rule for counting, the number of such 
paths is the product of the number of paths of length r from v; to v,, namely, b,,, and the number 
of edges from v; to v, namely, a,;. When these products are added for all possible intermediate 
vertices v,, the desired result follows by the sum rule for counting. < 


EXAMPLE 15 How many paths of length four are there from a to d in the simple graph G in Figure 8? 


¢ b Solution: The adjacency matrix of G (ordering the vertices as a, b, c, d) is 
0 1 1 0 
|1 0 0 1 
: ns 1 0 0 1f 
c 0 110 
FIGURE 8 The 
graph G. Hence, the number of paths of length four from a to d is the (1, 4)th entry of A4. Because 
8 0 0 8 
0 8 8 OF 
8 0 0 8 


Extra » there are exactly eight paths of length four from a to d. By inspection of the graph, we 
Examples see that a, b, a, b, d; a, b, a, c, d; a, b, d, b, d; a, b, d, c, d; a, c, a, b, d; a, c, a, c, d; a, c, d, b, d; and 
a, c, d, c, d are the eight paths of length four from a to d. < 


Theorem 2 can be used to find the length of the shortest path between two vertices of a 
graph (see Exercise 56), and it can also be used to determine whether a graph is connected (see 
Exercises 61 and 62). 


Exercises 


1. Does each of these lists of vertices form a path in the fol- a b c 
lowing graph? Which paths are simple? Which are cir- 
cuits? What are the lengths of those that are paths? 


a) a,e, b,c, b b) a,e,a,d,b,c,a 
c) e,b,a,d,b,e d) c, b,d,a,e,c 

d e 
a b c 


In Exercises 3-5 determine whether the given graph is 
connected. 


2. Does each of these lists of vertices form a path in the fol- 


lowing graph? Which paths are simple? Which are cir- 
cuits? What are the lengths of those that are paths? 


i ae Beeb b) ada d,a 4. 
c) a,d,b,e,a d) a,b,e,c,b,d,a 


10. 


11. 


12. 


. How many connected components does each of the 


graphs in Exercises 3-5 have? For each graph find each 
of its connected components. 


. What do the connected components of acquaintanceship 


graphs represent? 


. What do the connected components of a collaboration 


graph represent? 


. Explain why in the collaboration graph of mathemati- 


cians (see Example 3 in Section 10.1) a vertex represent- 
ing a mathematician is in the same connected component 
as the vertex representing Paul Erdős if and only if that 
mathematician has a finite Erdős number. 


In the Hollywood graph (see Example 3 in Section 10.1), 
when is the vertex representing an actor in the same 
connected component as the vertex representing Kevin 
Bacon? 


Determine whether each of these graphs is strongly con- 
nected and if not, whether it is weakly connected. 


a) a b c 
e d 
b) a b 
c 
e d 
c) 


Determine whether each of these graphs is strongly con- 
nected and if not, whether it is weakly connected. 


a) a b c 


13. 


14. 


15. 
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b) b 


s N 
(> 


What do the strongly connected components of a tele- 
phone call graph represent? 


Find the strongly connected components of each of these 
graphs. 


a) a b 


9 


e d 
b) a b c 
f A 
c) 
a d e 


E 


i T 


Find the strongly connected components of each of these 
graphs. 


TZ 


E c d 


n e 


g f 
c) a b C d 
i h g 


A 


e 
f 
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Suppose that G = (V, E) is a directed graph. A vertex w € V 
is reachable from a vertex v € V if there is a directed path 
from v to w. The vertices v and w are mutually reachable if 
there are both a directed path from v to w and a directed path 
from w to v in G. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


Show that if G = (V, E) is a directed graph and u, v, and 
w are vertices in V for which u and v are mutually reach- 
able and v and w are mutually reachable, then u and w are 
mutually reachable. 


Show that if G = (V, E) is a directed graph, then the 
strong components of two vertices u and v of V are ei- 
ther the same or disjoint. [Hint: Use Exercise 16.] 


Show that all vertices visited in a directed path connect- 
ing two vertices in the same strongly connected compo- 
nent of a directed graph are also in this strongly con- 
nected component. 


Find the number of paths of length n between two differ- 
ent vertices in K; if n is 


a) 2. b) 3. c) 4. d) 5. 


Use paths either to show that these graphs are not isomor- 
phic or to find an isomorphism between these graphs. 


uy Uy vi v 
U4 uz v4 v3 
G H 


Use paths either to show that these graphs are not isomor- 
phic or to find an isomorphism between them. 


uy Uy vi v 


ug U3 Vg V3 


u7 Ug V7 V4 


Use paths either to show that these graphs are not isomor- 
phic or to find an isomorphism between them. 


uy Uy vy v2 


ug U3 Vg V3 


23. 


24. 


25. 


26. 


27. 


*28. 


29. 


*30. 


Use paths either to show that these graphs are not isomor- 
phic or to find an isomorphism between them. 


uy u 


Avy O 
A 


G H 


Find the number of paths of length n between any two ad- 
jacent vertices in K3, for the values of n in Exercise 19. 


Find the number of paths of length n between any two 
nonadjacent vertices in K3, for the values of n in Exer- 
cise 19. 


Find the number of paths between c and d in the graph in 
Figure 1 of length 


a) 2. b) 3. c) 4. d) 5. e) 6. f)7. 


Find the number of paths from a to e in the directed graph 
in Exercise 2 of length 


a) 2. b) 3. c) 4. d) 5. e) 6. f)7. 


Show that every connected graph with n vertices has at 
least n — 1 edges. 

Let G = (V, E) be a simple graph. Let R be the relation 
on V consisting of pairs of vertices (u, v) such that there 
is a path from u to v or such that u = v. Show that R is an 
equivalence relation. 


Show that in every simple graph there is a path from every 
vertex of odd degree to some other vertex of odd degree. 


In Exercises 31-33 find all the cut vertices of the given graph. 


31. 


34. 
* 35, 


* 36. 


*37, 


a d e 32. a f 
7 y i, c e 
a b f 
e 
c & 
d i h 
Find all the cut edges in the graphs in Exercises 31-33. 


Suppose that v is an endpoint of a cut edge. Prove that v 
is a cut vertex if and only if this vertex is not pendant. 


Show that a vertex c in the connected simple graph G is 
a cut vertex if and only if there are vertices u and v, both 
different from c, such that every path between u and v 
passes through c. 


Show that a simple graph with at least two vertices has at 
least two vertices that are not cut vertices. 


* 38, 


39. 


Show that an edge in a simple graph is a cut edge if and 
only if this edge is not part of any simple circuit in the 
graph. 

A communications link in a network should be provided 
with a backup link if its failure makes it impossible for 
some message to be sent. For each of the communications 
networks shown here in (a) and (b), determine those links 
that should be backed up. 


a) Boston 


Chicago New York 


San Francisco 


Washington 
Los Angeles 
b) Bangor 
Burlington 
Seattle Portland 
Boston 
San Denver Chicago 


New York 
Washington 


\ 
Salt Lake Atlanta 
Los City 
Angeles 


A vertex basis in a directed graph G is a minimal set B of 
vertices of G such that for each vertex v of G not in B there is 
a path to v from some vertex B. 


40. 


41. 


42. 


* 43. 


* 44, 


* 45, 


46. 


Find a vertex basis for each of the directed graphs in 
Exercises 7-9 of Section 10.2. 

What is the significance of a vertex basis in an influence 
graph (described in Example 2 of Section 10.1)? Find a 
vertex basis in the influence graph in that example. 
Show that if a connected simple graph G is the union of 
the graphs G, and Gj, then G, and G, have at least one 
common vertex. 

Show that if a simple graph G has k connected compo- 
nents and these components have 1, n, ..., n, vertices, 
respectively, then the number of edges of G does not 


exceed 
k 
È Cm, 2). 
i=l 


Use Exercise 43 to show that a simple graph with n ver- 
tices and k connected components has at most (n — k)(n — 
k + 1)/2 edges. [Hint: First show that 


k 
bs <n? — (k -= 1)(2n — k), 
i=1 


where n; is the number of vertices in the ith connected 
component. ] 

Show that a simple graph G with n vertices is connected 
if it has more than (n — 1)(n — 2)/2 edges. 

Describe the adjacency matrix of a graph with n con- 
nected components when the vertices of the graph are 
listed so that vertices in each connected component are 
listed successively. 


47. 


48. 


49. 


50. 


51. 


52. 


53. 


54. 


* 55. 
56. 


57. 


58. 


ES" 59, 


60. 
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How many nonisomorphic connected simple graphs are 
there with n vertices when n is 

a) 2? b) 3? c) 4? d) 5? 

Show that each of the following graphs has no cut ver- 
tices. 

a) C, where n > 3 

b) W, where n > 3 

c) Knn where m > 2 andn > 2 

d) Q, where n > 2 

Show that each of the graphs in Exercise 48 has no cut 
edges. 

For each of these graphs, find x(G), A(G), and 
min,<y deg(v), and determine which of the two inequali- 
ties in x(G) < A(G) < min, ¢y deg(v) are strict. 


a) ef rs 


Show that if G is a connected graph, then it is possible to 
remove vertices to disconnect G if and only if G is not a 
complete graph. 

Show that if Gis a connected graph with n vertices then 
a) x(G) =n — 1 if and only ifG=K,,. 

b) A(G) =n — 1 if and only if G = K,,. 


Find x(K,,,,) and A(K,,,,,), where m and n are positive 
integers. 
Construct a graph G with x(G)= 1, A(G) = 2, and 


min,ey deg(v) = 3. 

Show that if G is a graph, then x(G) < A(G). 

Explain how Theorem 2 can be used to find the length of 
the shortest path from a vertex v to a vertex w in a graph. 
Use Theorem 2 to find the length of the shortest path be- 
tween a and f in the graph in Figure 1. 

Use Theorem 2 to find the length of the shortest path from 
a to c in the directed graph in Exercise 2. 

Let P, and P, be two simple paths between the vertices u 
and v in the simple graph G that do not contain the same 
set of edges. Show that there is a simple circuit in G. 
Show that the existence of a simple circuit of length k, 
where k is an integer greater than 2, is an invariant for 
graph isomorphism. 
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61. 


62. 


63. 


64. 
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Explain how Theorem 2 can be used to determine 
whether a graph is connected. 

Use Exercise 61 to show that the graph G, in Figure 2 
is connected whereas the graph G, in that figure is not 
connected. 

Show that a simple graph G is bipartite if and only if it 
has no circuits with an odd number of edges. 

In an old puzzle attributed to Alcuin of York (735-804), a 
farmer needs to carry a wolf, a goat, and a cabbage across 


c) Explain why finding a path from the vertex represent- 
ing (FWGC, Ø) to the vertex representing (Ø, FWGC ) 
solves the puzzle. 

d) Find two different solutions of the puzzle, each using 
seven crossings. 

e) Suppose that the farmer must pay a toll of one dollar 
whenever he crosses the river with an animal. Which 
solution of the puzzle should the farmer use to pay the 
least total toll? 


ariver. The farmer only has a small boat, which can carry *65. Use a graph model and a path in your graph, as in Exer- 
the farmer and only one object (an animal or a vegetable). cise 64, to solve the jealous husbands problem. Two 
He can cross the river repeatedly. However, if the farmer married couples, each a husband and a wife, want to 
is on the other shore, the wolf will eat the goat, and, simi- cross a river. They can only use a boat that can carry 
larly, the goat will eat the cabbage. We can describe each one or two people from one shore to the other shore. 
state by listing what is on each shore. For example, we Each husband is extremely jealous and is not willing 
can use the pair (FG, WC ) for the state where the farmer to leave his wife with the other husband, either in the 
and goat are on the first shore and the wolf and cabbage boat or on shore. How can these four people reach the 
are on the other shore. [The symbol Ø is used when noth- opposite shore? 

ing is on a shore, so that (FWGC, Ø) is the initial state. ] 66. Suppose that you have a three-gallon jug and a five-gallon 


a) Find all allowable states of the puzzle, where neither 
the wolf and the goat nor the goat and the cabbage are 
left on the same shore without the farmer. 

b) Construct a graph such that each vertex of this graph 
represents an allowable state and the vertices repre- 
senting two allowable states are connected by an edge 
if itis possible to move from one state to the other us- 
ing one trip of the boat. 


jug. You may fill either jug with water, you may empty 
either jug, and you may transfer water from either jug 
into the other jug. Use a path in a directed graph to 
show that you can end up with a jug containing exactly 
one gallon. [Hint: Use an ordered pair (a, b) to indicate 
how much water is in each jug. Represent these ordered 
pairs by vertices. Add an edge for each allowable opera- 
tion with the jugs.] 


Euler and Hamilton Paths 


Links » 


Only five bridges 
connect Kaliningrad 
today. Of these, just two 
remain from Euler’s day. 


10.5.1 Introduction 


Can we travel along the edges of a graph starting at a vertex and returning to it by traversing 
each edge of the graph exactly once? Similarly, can we travel along the edges of a graph starting 
at a vertex and returning to it while visiting each vertex of the graph exactly once? Although 
these questions seem to be similar, the first question, which asks whether a graph has an Euler 
circuit, can be easily answered simply by examining the degrees of the vertices of the graph, 
while the second question, which asks whether a graph has a Hamilton circuit, is quite difficult 
to solve for most graphs. In this section we will study these questions and discuss the difficulty 
of solving them. Although both questions have many practical applications in many different 
areas, both arose in old puzzles. We will learn about these old puzzles as well as modern 
practical applications. 


10.5.2 Euler Paths and Circuits 


The town of Königsberg, Prussia (now called Kaliningrad and part of the Russian republic), 
was divided into four sections by the branches of the Pregel River. These four sections included 
the two regions on the banks of the Pregel, Kneiphof Island, and the region between the two 
branches of the Pregel. In the eighteenth century seven bridges connected these regions. Figure 1 
depicts these regions and bridges. 

The townspeople took long walks through town on Sundays. They wondered whether it was 
possible to start at some location in the town, travel across all the bridges once without crossing 
any bridge twice, and return to the starting point. 
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C 
A D 
B 
FIGURE 1 The seven bridges of Königsberg. FIGURE 2 Multigraph model 


Definition 1 


EXAMPLE 1 


EXAMPLE 2 


of the town of Konigsberg. 


The Swiss mathematician Leonhard Euler solved this problem. His solution, published 
in 1736, may be the first use of graph theory. (For a translation of Euler’s original paper see 
[BiLIWi99].) Euler studied this problem using the multigraph obtained when the four regions 
are represented by vertices and the bridges by edges. This multigraph is shown in Figure 2. 

The problem of traveling across every bridge without crossing any bridge more than once 
can be rephrased in terms of this model. The question becomes: Is there a simple circuit in this 
multigraph that contains every edge? 


An Euler circuit in a graph G is a simple circuit containing every edge of G. An Euler path 
in G is a simple path containing every edge of G. 


Examples | and 2 illustrate the concept of Euler circuits and paths. 


Which of the undirected graphs in Figure 3 have an Euler circuit? Of those that do not, which 
have an Euler path? 


Solution: The graph G, has an Euler circuit, for example, a, e, c,d, e, b,a. Neither of the 
graphs G, or G, has an Euler circuit (the reader should verify this). However, G3 has 
an Euler path, namely, a, c,d, e,b,d,a,b. G, does not have an Euler path (as the reader 
should verify). < 


Which of the directed graphs in Figure 4 have an Euler circuit? Of those that do not, which have 
an Euler path? 


a b 
a b a b a b a b i : c d 
X | | KA | AI 
d c d c c d e d c ak a b 
e d 
A, A, A, 


Gi Gy 


G3 


FIGURE 3 The undirected graphs G,, G,, and G3. FIGURE 4 The directed graphs H,, H,, and H}. 
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Photo 


Solution: The graph H, has an Euler circuit, for example, a, g, c, b, g, e, d, f, a. Neither H} nor H, 
has an Euler circuit (as the reader should verify). H} has an Euler path, namely, c, a, b, c, d, b, 
but H; does not (as the reader should verify). < 


NECESSARY AND SUFFICIENT CONDITIONS FOR EULER CIRCUITS AND PATHS 
There are simple criteria for determining whether a multigraph has an Euler circuit or an Euler 
path. Euler discovered them when he solved the famous Königsberg bridge problem. We will 
assume that all graphs discussed in this section have a finite number of vertices and edges. 

What can we say if a connected multigraph has an Euler circuit? What we can show is that 
every vertex must have even degree. To do this, first note that an Euler circuit begins with a 
vertex a and continues with an edge incident with a, say {a, b}. The edge {a, b} contributes 
one to deg(a). Each time the circuit passes through a vertex it contributes two to the vertex’s 
degree, because the circuit enters via an edge incident with this vertex and leaves via another 
such edge. Finally, the circuit terminates where it started, contributing one to deg(a). There- 
fore, deg(a) must be even, because the circuit contributes one when it begins, one when it ends, 
and two every time it passes through a (if it ever does). A vertex other than a has even de- 
gree because the circuit contributes two to its degree each time it passes through the vertex. 
We conclude that if a connected graph has an Euler circuit, then every vertex must have even 
degree. 

Is this necessary condition for the existence of an Euler circuit also sufficient? That is, must 
an Euler circuit exist in a connected multigraph if all vertices have even degree? This question 
can be settled affirmatively with a construction. 

Suppose that G is a connected multigraph with at least two vertices and the degree of 
every vertex of G is even. We will form a simple circuit that begins at an arbitrary vertex a 
of G, building it edge by edge. Let x9 = a. First, we arbitrarily choose an edge {xọ, x, } inci- 
dent with a which is possible because G is connected. We continue by building a simple path 
{xo x1 b {Xp Xo}, -> {X,_1,%, }, successively adding edges one by one to the path until we can- 
not add another edge to the path. This happens when we reach a vertex for which we have already 
included all edges incident with that vertex in the path. For instance, in the graph G in Figure 5 
we begin at a and choose in succession the edges {a, f}, {f,c}, {c, b}, and {b, a}. 

The path we have constructed must terminate because the graph has a finite number of 
edges, so we are guaranteed to eventually reach a vertex for which no edges are available to add 
to the path. The path begins at a with an edge of the form {a, x}, and we now show that it must 
terminate at a with an edge of the form {y, a}. To see that the path must terminate at a, note 
that each time the path goes through a vertex with even degree, it uses only one edge to enter 


LEONHARD EULER (1707-1783) Leonhard Euler was the son of a Calvinist minister from the vicinity of 
Basel, Switzerland. At 13 he entered the University of Basel, pursuing a career in theology, as his father wished. 
At the university Euler was tutored by Johann Bernoulli of the famous Bernoulli family of mathematicians. 
His interest and skills led him to abandon his theological studies and take up mathematics. Euler obtained 
his master’s degree in philosophy at the age of 16. In 1727 Peter the Great invited him to join the Academy 
at St. Petersburg. In 1741 he moved to the Berlin Academy, where he stayed until 1766. He then returned to 
St. Petersburg, where he remained for the rest of his life. 

Euler was incredibly prolific, contributing to many areas of mathematics, including number theory, com- 
binatorics, and analysis, as well as its applications to such areas as music and naval architecture. He wrote 
over 1100 books and papers and left so much unpublished work that it took 47 years after he died for 
all his work to be published. During his life his papers accumulated so quickly that he kept a large pile 


of articles awaiting publication. The Berlin Academy published the papers on top of this pile so later results were often published 
before results they depended on or superseded. Euler had 13 children and was able to continue his work while a child or two bounced 
on his knees. He was blind for the last 17 years of his life, but because of his fantastic memory this did not diminish his mathematical 
output. The project of publishing his collected works, undertaken by the Swiss Society of Natural Science, is ongoing and will require 
more than 75 volumes. 


THEOREM 1 
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a a= <b 


[a d c d 
f ea = 
—~ 
e e 


G H 


FIGURE 5 Constructing an Euler circuit in G. 


this vertex, so because the degree must be at least two, at least one edge remains for the path 
to leave the vertex. Furthermore, every time we enter and leave a vertex of even degree, there 
are an even number of edges incident with this vertex that we have not yet used in our path. 
Consequently, as we form the path, every time we enter a vertex other than a, we can leave it. 
This means that the path can end only at a. Next, note that the path we have constructed may 
use all the edges of the graph, or it may not if we have returned to a for the last time before using 
all the edges. 

An Euler circuit has been constructed if all the edges have been used. Otherwise, consider 
the subgraph H obtained from G by deleting the edges already used and vertices that are not 
incident with any remaining edges. When we delete the circuit a, f, c, b, a from the graph in 
Figure 5, we obtain the subgraph labeled as H. 

Because G is connected, H has at least one vertex in common with the circuit that has been 
deleted. Let w be such a vertex. (In our example, c is the vertex.) 

Every vertex in H has even degree (because in G all vertices had even degree, and for each 
vertex, pairs of edges incident with this vertex have been deleted to form H). Note that H may 
not be connected. Beginning at w, construct a simple path in H by choosing edges as long as 
possible, as was done in G. This path must terminate at w. For instance, in Figure 5, c, d, e, c is 
a path in H. Next, form a circuit in G by splicing the circuit in H with the original circuit in G 
(this can be done because w is one of the vertices in this circuit). When this is done in the graph 
in Figure 5, we obtain the circuit a, f, c, d, e, c, b, a. 

Continue this process until all edges have been used. (The process must terminate because 
there are only a finite number of edges in the graph.) This produces an Euler circuit. The con- 
struction shows that if the vertices of a connected multigraph all have even degree, then the 
graph has an Euler circuit. 

We summarize these results in Theorem 1. 


A connected multigraph with at least two vertices has an Euler circuit if and only if each of 
its vertices has even degree. 


We can now solve the Königsberg bridge problem. Because the multigraph representing 
these bridges, shown in Figure 2, has four vertices of odd degree, it does not have an Euler 
circuit. There is no way to start at a given point, cross each bridge exactly once, and return to 
the starting point. 

Algorithm 1 gives the constructive procedure for finding Euler circuits given in the discus- 
sion preceding Theorem 1. (Because the circuits in the procedure are chosen arbitrarily, there 
is some ambiguity. We will not bother to remove this ambiguity by specifying the steps of the 
procedure more precisely.) 
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EXAMPLE 3 


ALGORITHM 1 Constructing Euler Circuits. 


procedure Euler(G: connected multigraph with all vertices of 
even degree) 
circuit := a circuit in G beginning at an arbitrarily chosen 
vertex with edges successively added to form a path that 
returns to this vertex 
H := G with the edges of this circuit removed 
while H has edges 
subcircuit := a circuit in H beginning at a vertex in H that 
also is an endpoint of an edge of circuit 
H := H with edges of subcircuit and all isolated vertices 
removed 
circuit := circuit with subcircuit inserted at the appropriate 
vertex 
return circuit {circuit is an Euler circuit} 


Algorithm 1 provides an efficient algorithm for finding Euler circuits in a connected multi- 
graph G with all vertices of even degree. We leave it to the reader (Exercise 66) to show that 
the worst case complexity of this algorithm is O(m), where m is the number of edges of G. 

Example 3 shows how Euler paths and circuits can be used to solve a type of puzzle. 


Many puzzles ask you to draw a picture in a continuous motion without lifting a pencil so that 
no part of the picture is retraced. We can solve such puzzles using Euler circuits and paths. 
For example, can Mohammed’s scimitars, shown in Figure 6, be drawn in this way, where the 
drawing begins and ends at the same point? 


Solution: We can solve this problem because the graph G shown in Figure 6 has an Euler circuit. 
It has such a circuit because all its vertices have even degree. We will use Algorithm 1 to con- 
struct an Euler circuit. First, we form the circuit a, b, d, c, b, e, i, f, e, a. We obtain the subgraph 
H by deleting the edges in this circuit and all vertices that become isolated when these edges 
are removed. Then we form the circuit d, g, h, j, i, h, k, g, f, d in H. After forming this circuit we 
have used all edges in G. Splicing this new circuit into the first circuit at the appropriate place 
produces the Euler circuit a, b, d, g, h, j, i, h, k, g, f, d, c, b, e, i, f, e, a. This circuit gives a way to 
draw the scimitars without lifting the pencil or retracing part of the picture. < 


Another algorithm for constructing Euler circuits, called Fleury’s algorithm, is described 
in the premble to Exercise 50. 

We will now show that a connected multigraph has an Euler path (and not an Euler circuit) if 
and only if it has exactly two vertices of odd degree. First, suppose that a connected multigraph 


G 


FIGURE 6 Mohammed’s scimitars. 


THEOREM 2 


EXAMPLE 4 


Links » 
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G, Gy G3 
FIGURE 7 Three undirected graphs. 


does have an Euler path from a to b, but not an Euler circuit. The first edge of the path contributes 
one to the degree ofa. A contribution of two to the degree of a is made every time the path passes 
through a. The last edge in the path contributes one to the degree of b. Every time the path 
goes through b there is a contribution of two to its degree. Consequently, both a and b have odd 
degree. Every other vertex has even degree, because the path contributes two to the degree of a 
vertex whenever it passes through it. 

Now consider the converse. Suppose that a graph has exactly two vertices of odd degree, 
say a and b. Consider the larger graph made up of the original graph with the addition of an 
edge {a, b}. Every vertex of this larger graph has even degree, so there is an Euler circuit. The 
removal of the new edge produces an Euler path in the original graph. Theorem 2 summarizes 
these results. 


A connected multigraph has an Euler path but not an Euler circuit if and only if it has exactly 
two vertices of odd degree. 


Which graphs shown in Figure 7 have an Euler path? 


Solution: G, contains exactly two vertices of odd degree, namely, b and d. Hence, it has an Euler 
path that must have b and d as its endpoints. One such Euler path is d, a, b, c, d, b. Similarly, G» 
has exactly two vertices of odd degree, namely, b and d. So it has an Euler path that must have 
b and d as endpoints. One such Euler path is b, a, g, f, e, d, c, g, b, c, f, d. G, has no Euler path 
because it has six vertices of odd degree. < 


Returning to eighteenth-century Königsberg, is it possible to start at some point in the town, 
travel across all the bridges, and end up at some other point in town? This question can be 
answered by determining whether there is an Euler path in the multigraph representing the 
bridges in Königsberg. Because there are four vertices of odd degree in this multigraph, there 
is no Euler path, so such a trip is impossible. 

Necessary and sufficient conditions for Euler paths and circuits in directed graphs are given 
in Exercises 16 and 17. 


APPLICATIONS OF EULER PATHS AND CIRCUITS Euler paths and circuits can be used 
to solve many practical problems. For example, many applications ask for a path or circuit that 
traverses each street in a neighborhood, each road in a transportation network, each connection 
in a utility grid, or each link in a communications network exactly once. Finding an Euler path 
or circuit in the appropriate graph model can solve such problems. For example, if a postman 
can find an Euler path in the graph that represents the streets the postman needs to cover, this 
path produces a route that traverses each street of the route exactly once. If no Euler path exists, 
some streets will have to be traversed more than once. The problem of finding a circuit in a graph 
with the fewest edges that traverses every edge at least once is known as the Chinese postman 
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problem in honor of Guan Meigu, who posed it in 1962. See [MiRo91] for more information on 
the solution of the Chinese postman problem when no Euler path exists. 

Among the other areas where Euler circuits and paths are applied is in the layout of circuits, 
in network multicasting, and in molecular biology, where Euler paths are used in the sequencing 
of DNA. 


10.5.3 Hamilton Paths and Circuits 


We have developed necessary and sufficient conditions for the existence of paths and circuits 
that contain every edge of a multigraph exactly once. Can we do the same for simple paths and 
circuits that contain every vertex of the graph exactly once? 


A simple path in a graph G that passes through every vertex exactly once is called a Hamilton 
path, and a simple circuit in a graph G that passes through every vertex exactly once is called 
a Hamilton circuit. That is, the simple path xo, Xi, ...,x,_, Xn in the graph G = (V, E) is a 
Hamilton path if V = {xo, xy, -- -> Xn- Xn} and x; # x; for O < i < j < n, and the simple circuit 
Xo, Xs a a Xp» XQ (with n > 0) is a Hamilton circuit if xp, x1, ...,X,_1,X, 18 a Hamilton 
path. 


This terminology comes from a game, called the Icosian puzzle, invented in 1857 by the 
Irish mathematician Sir William Rowan Hamilton. It consisted of a wooden dodecahedron [a 
polyhedron with 12 regular pentagons as faces, as shown in Figure 8(a)], with a peg at each 
vertex of the dodecahedron, and string. The 20 vertices of the dodecahedron were labeled with 
different cities in the world. The object of the puzzle was to start at a city and travel along the 
edges of the dodecahedron, visiting each of the other 19 cities exactly once, and end back at the 
first city. The circuit traveled was marked off using the string and pegs. 

Because the author cannot supply each reader with a wooden solid with pegs and string, we 
will consider the equivalent question: Is there a circuit in the graph shown in Figure 8(b) that 
passes through each vertex exactly once? This solves the puzzle because this graph is isomorphic 
to the graph consisting of the vertices and edges of the dodecahedron. A solution of Hamilton’s 
puzzle is shown in Figure 9. 


Shoe 
a 


(a) (b) 
FIGURE 9 A solution to 
FIGURE 8 Hamilton’s “A Voyage Round the the “A Voyage Round the 
World” puzzle. World” puzzle. 


Which of the simple graphs in Figure 10 have a Hamilton circuit or, if not, a Hamilton path? 


Solution: G, has a Hamilton circuit: a, b, c, d, e, a. There is no Hamilton circuit in G, (this can 
be seen by noting that any circuit containing every vertex must contain the edge {a, b} twice), 
but G, does have a Hamilton path, namely, a, b, c, d. G} has neither a Hamilton circuit nor a 
Hamilton path, because any path containing all vertices must contain one of the edges {a, b}, 
{e, f}, and {c, d} more than once. 
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FIGURE 10 Three simple graphs. 


CONDITIONS FOR THE EXISTENCE OF HAMILTON CIRCUITS Is there a simple way 
to determine whether a graph has a Hamilton circuit or path? At first, it might seem that there 
should be an easy way to determine this, because there is a simple way to answer the similar 
question of whether a graph has an Euler circuit. Surprisingly, there are no known simple nec- 
essary and sufficient criteria for the existence of Hamilton circuits. However, many theorems 
are known that give sufficient conditions for the existence of Hamilton circuits. Also, certain 
properties can be used to show that a graph has no Hamilton circuit. For instance, a graph with a 
vertex of degree one cannot have a Hamilton circuit, because in a Hamilton circuit, each vertex 
is incident with two edges in the circuit. Moreover, if a vertex in the graph has degree two, then 
both edges that are incident with this vertex must be part of any Hamilton circuit. Also, note that 
when a Hamilton circuit is being constructed and this circuit has passed through a vertex, then all 
remaining edges incident with this vertex, other than the two used in the circuit, can be removed 
from consideration. Furthermore, a Hamilton circuit cannot contain a smaller circuit within it. 


EXAMPLE 6 Show that neither graph displayed in Figure 11 has a Hamilton circuit. 


Solution: There is no Hamilton circuit in G because G has a vertex of degree one, namely, e. Now 
consider H. Because the degrees of the vertices a, b, d, and e are all two, every edge incident 
with these vertices must be part of any Hamilton circuit. It is now easy to see that no Hamilton 
circuit can exist in H, for any Hamilton circuit would have to contain four edges incident with 
c, which is impossible. < 


WILLIAM ROWAN HAMILTON (1805-1865) William Rowan Hamilton, the most famous Irish scientist 
ever to have lived, was born in 1805 in Dublin. His father was a successful lawyer, his mother came from a 
family noted for their intelligence, and he was a child prodigy. By the age of 3 he was an excellent reader 
and had mastered advanced arithmetic. Because of his brilliance, he was sent off to live with his uncle James, 
a noted linguist. By age 8 Hamilton had learned Latin, Greek, and Hebrew; by 10 he had also learned Ital- 
ian and French and he began his study of oriental languages, including Arabic, Sanskrit, and Persian. Dur- 
ing this period he took pride in knowing as many languages as his age. At 17, no longer devoted to learn- 
ing new languages and having mastered calculus and much mathematical astronomy, he began original work 
©Hulton-Deutsch/ in optics, and he also found an important mistake in Laplace’s work on celestial mechanics. Before enter- 
Corbis/Getty Images ing Trinity College, Dublin, at 18, Hamilton had not attended school; rather, he received private tutoring. At 

Trinity, he was a superior student in both the sciences and the classics. Prior to receiving his degree, because 
of his brilliance he was appointed the Astronomer Royal of Ireland, beating out several famous astronomers for the post. He held this 
position until his death, living and working at Dunsink Observatory outside of Dublin. Hamilton made important contributions to 
optics, abstract algebra, and dynamics. Hamilton invented algebraic objects called quaternions as an example of a noncommutative 
system. He discovered the appropriate way to multiply quaternions while walking along a canal in Dublin. In his excitement, he 
carved the formula in the stone of a bridge crossing the canal, a spot marked today by a plaque. Later, Hamilton remained obsessed 
with quaternions, working to apply them to other areas of mathematics, instead of moving to new areas of research. 

In 1857 Hamilton invented “The Icosian Game” based on his work in noncommutative algebra. He sold the idea for 25 pounds 
to a dealer in games and puzzles. (Because the game never sold well, this turned out to be a bad investment for the dealer.) The 
“Traveler’s Dodecahedron,” also called “A Voyage Round the World,” the puzzle described in this section, is a variant of that game. 

Hamilton married his third love in 1833, but his marriage worked out poorly, because his wife, a semi-invalid, was unable to 
cope with his household affairs. He suffered from alcoholism and lived reclusively for the last two decades of his life. He died from 
gout in 1865, leaving masses of papers containing unpublished research. Mixed in with these papers were a large number of dinner 
plates, many containing the remains of desiccated, uneaten chops. 
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FIGURE 11 Two graphs that do not have a Hamilton circuit. 


Show that K, has a Hamilton circuit whenever n > 3. 


Solution: We can form a Hamilton circuit in K,, beginning at any vertex. Such a circuit can be 
built by visiting vertices in any order we choose, as long as the path begins and ends at the same 
vertex and visits each other vertex exactly once. This is possible because there are edges in K, 
between any two vertices. < 


Although no useful necessary and sufficient conditions for the existence of Hamilton cir- 
cuits are known, quite a few sufficient conditions have been found. Note that the more edges a 
graph has, the more likely it is to have a Hamilton circuit. Furthermore, adding edges (but not 
vertices) to a graph with a Hamilton circuit produces a graph with the same Hamilton circuit. 
So as we add edges to a graph, especially when we make sure to add edges to each vertex, we 
make it increasingly likely that a Hamilton circuit exists in this graph. Consequently, we would 
expect there to be sufficient conditions for the existence of Hamilton circuits that depend on 
the degrees of vertices being sufficiently large. We state two of the most important sufficient 
conditions here. These conditions were found by Gabriel A. Dirac in 1952 and Øystein Ore 
in 1960. 


DIRAC’S THEOREM If G is a simple graph with n vertices with n > 3 such that the 
degree of every vertex in G is at least n/2, then G has a Hamilton circuit. 


ORE’S THEOREM If G is a simple graph with n vertices with n > 3 such that 
deg(u) + deg(v) > n for every pair of nonadjacent vertices u and v in G, then G has a 
Hamilton circuit. 


The proof of Ore’s theorem is outlined in Exercise 65. Dirac’s theorem can be proved as a corol- 
lary to Ore’s theorem because the conditions of Dirac’s theorem imply those of Ore’s theorem. 

Both Ore’s theorem and Dirac’s theorem provide sufficient conditions for a connected sim- 
ple graph to have a Hamilton circuit. However, these theorems do not provide necessary condi- 
tions for the existence of a Hamilton circuit. For example, the graph C; has a Hamilton circuit 
but does not satisfy the hypotheses of either Ore’s theorem or Dirac’s theorem, as the reader 
can verify. 

The best algorithms known for finding a Hamilton circuit in a graph or determining that 
no such circuit exists have exponential worst-case time complexity (in the number of vertices 
of the graph). Finding an algorithm that solves this problem with polynomial worst-case time 
complexity would be a major accomplishment because it has been shown that this problem is 
NP-complete (see Section 3.3). Consequently, the existence of such an algorithm would imply 
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that many other seemingly intractable problems could be solved using algorithms with polyno- 
mial worst-case time complexity. 


10.5.4 Applications of Hamilton Circuits 


Hamilton paths and circuits can be used to solve practical problems. For example, many appli- 
cations ask for a path or circuit that visits each road intersection in a city, each place pipelines 
intersect in a utility grid, or each node in a communications network exactly once. Finding 
a Hamilton path or circuit in the appropriate graph model can solve such problems. The fa- 
mous traveling salesperson problem or TSP (also known in older literature as the travel- 
ing salesman problem) asks for the shortest route a traveling salesperson should take to visit 
a set of cities. This problem reduces to finding a Hamilton circuit in a complete graph such 
that the total weight of its edges is as small as possible. We will return to this question in 
Section 10.6. 
We now describe a less obvious application of Hamilton circuits to coding. 


Gray Codes The position of a rotating pointer can be represented in digital form. One way to 
do this is to split the circle into 2” arcs of equal length and to assign a bit string of length n to 
each arc. Two ways to do this using bit strings of length three are shown in Figure 12. 

The digital representation of the position of the pointer can be determined using a set of n 
contacts. Each contact is used to read one bit in the digital representation of the position. This 
is illustrated in Figure 13 for the two assignments from Figure 12. 

When the pointer is near the boundary of two arcs, a mistake may be made in reading its 
position. This may result in a major error in the bit string read. For instance, in the coding 
scheme in Figure 12(a), if a small error is made in determining the position of the pointer, the 
bit string 100 is read instead of 011. All three bits are incorrect! To minimize the effect of an 


GABRIEL ANDREW DIRAC (1925-1984) Gabriel Dirac was born in Budapest. He moved to England in 
1937 when his mother married the illustrious physicist and Nobel Laureate Paul Adrien Maurice Dirac, who 
adopted him. Gabriel A. Dirac entered Cambridge University in 1942, but his studies were interrupted by 
wartime service in the aviation industry. He obtained his Ph.D. in mathematics in 1951 from the University 
of London. He held university positions in England, Canada, Austria, Germany, and Denmark, where he spent 
his last 14 years. Dirac became interested in graph theory early in his career and help raise its status as an im- 
portant topic of research. He made important contributions to many aspects of graph theory, including graph 
coloring and Hamilton circuits. Dirac attracted many students to graph theory and was noted as an excellent 
lecturer. 

Dirac was noted for his penetrating mind and held unconventional views on many topics, including politics 


and social life. Dirac was a man with many interests and held a great passion for fine art. He had a happy family life with his wife 
Rosemari and his four children. 


Courtesy of Museum of 
University History (MUV) 


ØYSTEIN ORE (1899-1968) Ore was born in Kristiania (the old name for Oslo, Norway). In 1922 he received 
his bachelors degree and in 1925 his Ph.D. in mathematics from Kristiania University, after studies in Germany 
and in Sweden. In 1927 he was recruited to leave his junior position at Kristiania and join Yale University. He 
was promoted rapidly at Yale, becoming full professor in 1929 and Sterling Professor in 1931, a position he 
held until 1968. 

Ore made many contributions to number theory, ring theory, lattice theory, graph theory, and probability 
theory. He was a prolific author of papers and books. His interest in the history of mathematics is reflected in 
his biographies of Abel and Cardano, and in his popular textbook Number Theory and its History. He wrote 
four books on graph theory in the 1960s. 

During and after World War II Ore played a major role supporting his native Norway. In 1947 
King Haakon VII of Norway gave him the Knight Order of St. Olaf to recognize these efforts. 


Ore possessed deep knowledge of painting and sculpture and was an ardent collector of ancient maps. He was married and had two 


children. 
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FIGURE 12 Converting the position of a pointer into digital form. 


error in determining the position of the pointer, the assignment of the bit strings to the 2” arcs 
should be made so that only one bit is different in the bit strings represented by adjacent arcs. 
This is exactly the situation in the coding scheme in Figure 12(b). An error in determining the 
position of the pointer gives the bit string 010 instead of 011. Only one bit is wrong. 

A Gray code is a labeling of the arcs of the circle such that adjacent arcs are labeled with bit 
strings that differ in exactly one bit. The assignment in Figure 12(b) is a Gray code. We can find 
a Gray code by listing all bit strings of length n in such a way that each string differs in exactly 
one position from the preceding bit string, and the last string differs from the first in exactly one 
position. We can model this problem using the n-cube Q,,. What is needed to solve this problem 
is a Hamilton circuit in Q,. Such Hamilton circuits are easily found. For instance, a Hamilton 
circuit for Q, is displayed in Figure 14. The sequence of bit strings differing in exactly one bit 
produced by this Hamilton circuit is 000, 001, 011, 010, 110, 111, 101, 100. 


Third bit is 1 here 


Second bit is 1 here Third bit is 1 here 


DI 


Second bit is 1 here 


Third bit is 1 here 


Third bit is 1 here Q Third bit is 1 here 


First bit is 1 here = 


Third bit is 1 here 


First bit is 1 here 


Second bit is 1 here 000 001 


FIGURE 13 The digital representation of the position FIGURE 14 A Hamilton 
of the pointer. circuit for Q}. 
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Gray codes are named after Frank Gray, who invented them in the 1940s at AT&T Bell 
Laboratories to minimize the effect of errors in transmitting digital signals. < 


Exercises 


In Exercises 1-8 determine whether the given graph has an 7. 
Euler circuit. Construct such a circuit when one exists. If 

no Euler circuit exists, determine whether the graph has an 

Euler path and construct such a path if one exists. 


aa f 
D k I m n o 


I 
9. Suppose that in addition to the seven bridges of 
Königsberg (shown in Figure 1) there were two addi- 
3. a b 4. a tional bridges, connecting regions B and C and regions 
B and D, respectively. Could someone cross all nine 
wv of these bridges exactly once and return to the starting 
b point? 
10. Can someone cross all the bridges shown in this map ex- 
actly once and return to the starting point? 
c d 
e C 
5 a b 


11. When can the centerlines of the streets in a city be painted 
without traveling a street more than once? (Assume that 
all the streets are two-way streets.) 

f 12. Devise a procedure, similar to Algorithm 1, for construct- 

ing Euler paths in multigraphs. 
e In Exercises 13—15 determine whether the picture shown can 
be drawn with a pencil in a continuous motion without lifting 


the pencil or retracing part of the picture. 
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Show that a directed multigraph having no isolated ver- 
tices has an Euler circuit if and only if the graph is weakly 
connected and the in-degree and out-degree of each ver- 
tex are equal. 


Show that a directed multigraph having no isolated ver- 
tices has an Euler path but not an Euler circuit if and 
only if the graph is weakly connected and the in-degree 
and out-degree of each vertex are equal for all but two 
vertices, one that has in-degree one larger than its out- 
degree and the other that has out-degree one larger than 
its in-degree. 


In Exercises 18-23 determine whether the directed graph 
shown has an Euler circuit. Construct an Euler circuit if one 
exists. If no Euler circuit exists, determine whether the di- 
rected graph has an Euler path. Construct an Euler path if one 


exists. 
18. a b 19. a b 
c > d d È 
20. a b c 
~e | + 2 
d e 
21. b 


23. 


*24. 


25. 


26. 


27. 


28. 


29. 


a b c 
d < f 
h 
g8 l 
j k l 
Devise an algorithm for constructing Euler circuits in di- 


rected graphs. 


Devise an algorithm for constructing Euler paths in di- 
rected graphs. 


For which values of n do these graphs have an Euler cir- 
cuit? 
a) K,, b) C, c) W, d) Q, 


For which values of n do the graphs in Exercise 26 have 
an Euler path but no Euler circuit? 


For which values of m and n does the complete bipartite 
graph K,,,,, have an 


a) Euler circuit? 
b) Euler path? 


Find the least number of times it is necessary to lift a 
pencil from the paper when drawing each of the graphs 
in Exercises 1—7 without retracing any part of the graph. 


In Exercises 30-36 determine whether the given graph has a 
Hamilton circuit. If it does, find such a circuit. If it does not, 
give an argument to show why no such circuit exists. 
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33. 
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34. 


35. 


36. 


37. 


38. 
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40. 


*41. 


42. 


43. 


44. 


45. 


*46. 


a b c 
$ > $ 
i d k 
d¢— Py _4 oh 
n l 
m 
e è è 
e f g 
a b 
c e 
a b c 
d S tf 
g h i 
Does the graph in Exercise 30 have a Hamilton path? If 


so, find such a path. If it does not, give an argument to 
show why no such path exists. 


Does the graph in Exercise 31 have a Hamilton path? If 
so, find such a path. If it does not, give an argument to 
show why no such path exists. 


Does the graph in Exercise 32 have a Hamilton path? If 
so, find such a path. If it does not, give an argument to 
show why no such path exists. 


Does the graph in Exercise 33 have a Hamilton path? If 
so, find such a path. If it does not, give an argument to 
show why no such path exists. 


Does the graph in Exercise 34 have a Hamilton path? If 
so, find such a path. If it does not, give an argument to 
show why no such path exists. 


Does the graph in Exercise 35 have a Hamilton path? If 
so, find such a path. If it does not, give an argument to 
show why no such path exists. 


Does the graph in Exercise 36 have a Hamilton path? If 
so, find such a path. If it does not, give an argument to 
show why no such path exists. 


For which values of n do the graphs in Exercise 26 have 
a Hamilton circuit? 


For which values of m and n does the complete bipartite 
graph K,,,,, have a Hamilton circuit? 


mn 
Show that the Petersen graph, shown here, does not have 
a Hamilton circuit, but that the subgraph obtained by 
deleting a vertex v, and all edges incident with v, does 
have a Hamilton circuit. 


47. 


48. 


* 49, 
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For each of these graphs, determine (i) whether Dirac’s 
theorem can be used to show that the graph has a Hamil- 
ton circuit, (ii) whether Ore’s theorem can be used to 
show that the graph has a Hamilton circuit, and (iii) 
whether the graph has a Hamilton circuit. 


a) À b) 
e—_____® 


c) d) 


Can you find a simple graph with n vertices with n > 3 
that does not have a Hamilton circuit, yet the degree of 
every vertex in the graph is at least (n — 1)/2? 

Show that there is a Gray code of order n whenever n 
is a positive integer, or equivalently, show that the n- 
cube Q,„ n > 1, always has a Hamilton circuit. [Hint: Use 
mathematical induction. Show how to produce a Gray 
code of order n from one of order n — 1.] 


» Fleury’s algorithm, published in 1883, constructs Euler cir- 
cuits by first choosing an arbitrary vertex of a connected 
multigraph, and then forming a circuit by choosing edges suc- 
cessively. Once an edge is chosen, it is removed. Edges are 
chosen successively so that each edge begins where the last 
edge ends, and so that this edge is not a cut edge unless there 
is no alternative. 


50. 


* 51. 
** 52, 


*53, 


54. 


55. 


Use Fleury’s algorithm to find an Euler circuit in the 
graph G in Figure 5. 

Express Fleury’s algorithm in pseudocode. 

Prove that Fleury’s algorithm always produces an Euler 
circuit. 

Give a variant of Fleury’s algorithm to produce Euler 
paths. 

A diagnostic message can be sent out over a computer 
network to perform tests over all links and in all devices. 
What sort of paths should be used to test all links? To test 
all devices? 

Show that a bipartite graph with an odd number of ver- 
tices does not have a Hamilton circuit. 
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A knight is a chess piece that can move either two spaces hor- * 59, Show that there is a knight’s tour on a 3 x 4 chessboard. 
izọntally añid one space verucally e aes horizontally *60. Show that there is no knight’s tour on a 3 x 3 chessboard. 
and two spaces vertically. That is, a knight on square (x, y) can 

move to any of the eight squares (x + 2, y+ 1), œ 1, y +2), * 61. Show that there is no knight’s tour on a 4 x 4 chessboard. 
if these squares are on the chessboard, as illustrated here. 62. Show that the graph representing the legal moves of a 


knight on an m Xn chessboard, whenever m and n are 
positive integers, is bipartite. 


63. Show that there is no reentrant knight’s tour on an m x n 
chessboard when m and n are both odd. [Hint: Use Exer- 
cises 55, 58b, and 62.] 


* 64. Show that there is a knight’s tour on an 8 x 8 chessboard. 
[Hint: You can construct a knight’s tour using a method 
invented by H. C. Warnsdorff in 1823: Start in any square, 
and then always move to a square connected to the fewest 
number of unused squares. Although this method may not 


A knight’s tour is a sequence of legal moves by a knight start- always produce a knight’s tour, it often does. ] 

ing at some square and visiting each square exactly once. A 65. The parts of this exercise outline a proof of Ore’s the- 
knight’s tour is called reentrant if there is a legal move that orem. Suppose that G is a simple graph with n vertices, 
takes the knight from the last square of the tour back to where n > 3, and deg(x) + deg(y) > n whenever x and y are non- 
the tour began. We can model knight’s tours using the graph adjacent vertices in G. Ore’s theorem states that under 
that has a vertex for each square on the board, with an edge these conditions, G has a Hamilton circuit. 


connecting two vertices if a knight can legally move between 
the squares represented by these vertices. 


56. 


57. 


58. 


a) Show that if G does not have a Hamilton circuit, then 
there exists another graph H with the same vertices 
as G, which can be constructed by adding edges to G, 
such that the addition of a single edge would produce 
a Hamilton circuit in H. [Hint: Add as many edges as 
possible at each successive vertex of G without pro- 
ducing a Hamilton circuit. ] 


b) Show that there is a Hamilton path in H. 


Draw the graph that represents the legal moves of a knight 

on a 3 xX 3 chessboard. 

Draw the graph that represents the legal moves of a knight 

on a 3 X 4 chessboard. 

a) Show that finding a knight’s tour on an m X n chess- 
board is equivalent to finding a Hamilton path on the 
graph representing the legal moves of a knight on that 


board. c) Let vi, v,...,¥, be a Hamilton path in H. Show 

b) Show that finding a reentrant knight’s tour on an that deg(v,) + deg(v,,) > n and that there are at most 
m X n chessboard is equivalent to finding a Hamilton deg(v,) vertices not adjacent to v, (including v, it- 
circuit on the corresponding graph. self). 


JULIUS PETER CHRISTIAN PETERSEN (1839-1910) Julius Petersen was born in the Danish town of Sorg. 
His father was a dyer. In 1854 his parents were no longer able to pay for his schooling, so he became an appren- 
tice in an uncle’s grocery store. When this uncle died, he left Petersen enough money to return to school. After 
graduating, he began studying engineering at the Polytechnical School in Copenhagen, later deciding to con- 
centrate on mathematics. He published his first textbook, a book on logarithms, in 1858. When his inheritance 
ran out, he had to teach to make a living. From 1859 until 1871 Petersen taught at a prestigious private high 
school in Copenhagen. While teaching high school he continued his studies, entering Copenhagen University 
in 1862. He married Laura Bertelsen in 1862; they had three children, two sons and a daughter. 
©Paul Fearn/Alamy Stock Petersen obtained a mathematics degree from Copenhagen University in 1866 and finally obtained his 
Photo doctorate in 1871 from that school. After receiving his doctorate, he taught at a polytechnic and military 
academy. In 1887 he was appointed to a professorship at the University of Copenhagen. Petersen was well known 

in Denmark as the author of a large series of textbooks for high schools and universities. One of his books, Methods and Theories 
for the Solution of Problems of Geometrical Construction, was translated into eight languages, with the English language version 
last reprinted in 1960 and the French version reprinted as recently as 1990, more than a century after the original publication date. 

Petersen worked in a wide range of areas, including algebra, analysis, cryptography, geometry, mechanics, mathematical eco- 
nomics, and number theory. His contributions to graph theory, including results on regular graphs, are his best-known work. He 
was noted for his clarity of exposition, problem-solving skills, originality, sense of humor, vigor, and teaching. One interesting fact 
about Petersen was that he preferred not to read the writings of other mathematicians. This led him often to rediscover results already 
proved by others, often with embarrassing consequences. However, he was often angry when other mathematicians did not read his 
writings! 

Petersen’s death was front-page news in Copenhagen. A newspaper of the time described him as the Hans Christian Andersen 
of science—a child of the people who made good in the academic world. 
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d) Let S be the set of vertices preceding each vertex adja- * 67. Show that this graph does not have a Hamilton circuit. 
cent to v, in the Hamilton path. Show that S contains 
deg(v,) vertices and v, É S. 


e) Show that S contains a vertex v, that is adjacent to v,,, 
implying that there are edges connecting v; and v,,, 
and v, and v,,. 


f) Show that part (e) implies that vi, v5,...,V%_1, 


Vis Vw Vn-p <--> Veg Yı is a Hamilton circuit in G. 
Conclude from this contradiction that Ore’s theorem 
holds. 

66. Show that if u and v are nonadjacent vertices in a graph * 68. Show that the worst case computational complexity of 
G with n vertices and deg(u) + deg(v) > n, then G has a Algorithm 1 for finding Euler circuits in a connected 
Hamilton circuit if and only if G + {u, v} has a Hamilton graph with all vertices of even degree is O(m), where m 
circuit. is the number of edges of G. 


Shortest-Path Problems 


10.6.1 Introduction 


Many problems can be modeled using graphs with weights assigned to their edges. As an il- 
lustration, consider how an airline system can be modeled. We set up the basic graph model by 
representing cities by vertices and flights by edges. Problems involving distances can be mod- 
eled by assigning distances between cities to the edges. Problems involving flight time can be 
modeled by assigning flight times to edges. Problems involving fares can be modeled by assign- 
ing fares to the edges. Figure 1 displays three different assignments of weights to the edges of a 
graph representing distances, flight times, and fares, respectively. 

Graphs that have a number assigned to each edge are called weighted graphs. Weighted 
graphs are used to model computer networks. Communications costs (such as the monthly cost 
of leasing a telephone line), the response times of the computers over these lines, or the distance 
between computers, can all be studied using weighted graphs. Figure 2 displays weighted graphs 
that represent three ways to assign weights to the edges of a graph of a computer network, 
corresponding to distance, response time, and cost. 

Several types of problems involving weighted graphs arise frequently. Determining a path 
of least length between two vertices in a network is one such problem. To be more specific, let 
the length of a path in a weighted graph be the sum of the weights of the edges of this path. (The 
reader should note that this use of the term length is different from the use of length to denote the 
number of edges in a path in a graph without weights.) The question is: What is a shortest path, 
that is, a path of least length, between two given vertices? For instance, in the airline system 
represented by the weighted graph shown in Figure 1, what is a shortest path in air distance 
between Boston and Los Angeles? What combinations of flights has the smallest total flight 
time (that is, total time in the air, not including time between flights) between Boston and Los 
Angeles? What is the cheapest fare between these two cities? In the computer network shown in 
Figure 2, what is a least expensive set of telephone lines needed to connect the computers in San 
Francisco with those in New York? Which set of telephone lines gives a fastest response time 
for communications between San Francisco and New York? Which set of lines has a shortest 
overall distance? 

Another important problem involving weighted graphs asks for a circuit of shortest total 
length that visits every vertex of a complete graph exactly once. This is the famous traveling 
salesperson problem, which asks for an order in which a salesperson should visit each of the 
cities on his route exactly once so that he travels the minimum total distance. We will discuss 
the traveling salesperson problem later in this section. 
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FIGURE 1 Weighted graphs modeling an airline system. 


10.6.2 A Shortest-Path Algorithm 


There are several different algorithms that find a shortest path between two vertices in a weighted 
graph. We will present a greedy algorithm discovered by the Dutch mathematician Edsger 
Dijkstra in 1959. The version we will describe solves this problem in undirected weighted graphs 
where all the weights are positive. It is easy to adapt it to solve shortest-path problems in directed 
graphs. 

Before giving a formal presentation of the algorithm, we will give an illustrative example. 


What is the length of a shortest path between a and z in the weighted graph shown in Figure 3? 
Solution: Although a shortest path is easily found by inspection, we will develop some ideas 


useful in understanding Dijkstra’s algorithm. We will solve this problem by finding the length 
of a shortest path from a to successive vertices, until z is reached. 
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FIGURE 2 Weighted graphs modeling a computer network. 


FIGURE 3 A weighted simple graph. 


A path that starts at a that has no interior vertices other than a must consist of one edge 
that has a as one of its endpoints. These are the one-edged paths a, b of length 4 and a, d of 
length 2. It follows that d is the closest vertex to a, and the shortest path from a to d has 
length 2. 

We can find the second closest vertex by examining all paths that begin with the shortest 
path from a to a vertex in the set {a, d}, followed by an edge that has one endpoint in {a, d} and 
its other endpoint not in this set. There are two such paths to consider, a, d, e of length 7 and 
a, b of length 4. Hence, the second closest vertex to a is b, and the shortest path from a to b has 
length 4. 
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To find the third closest vertex to a, we need examine only the paths that begin with the 
shortest path from a to a vertex in the set {a, d, b}, followed by an edge that has one endpoint 
in the set {a, d, b} and its other endpoint not in this set. There are three such paths, a, b, c of 
length 7, a, b, e of length 7, and a, d, e of length 5. Because the shortest of these paths is a, d, e, 
the third closest vertex to a is e, and the length of the shortest path from a to e is 5. 

To find the fourth closest vertex to a, we need examine only the paths that begin with the 
shortest path from a to a vertex in the set {a, d, b, e}, followed by an edge that has one endpoint 
in the set {a, d, b, e} and its other endpoint not in this set. There are two such paths, a, b, c of 
length 7 and a, d, e, z of length 6. Because the shorter of these paths is a, d, e, z, the fourth closest 
vertex to a is z, and the length of the shortest path from a to z is 6. < 


Example 1 illustrates the general principles used in Dijkstra’s algorithm. Note that a shortest 
path from a to z could have been found by a brute force approach by examining the length of 
every path from a to z. However, this brute force approach is impractical for humans and even 
for computers for graphs with a large number of edges. 

We will now consider the general problem of finding the length of a shortest path between 
a and z in an undirected connected simple weighted graph. Dijkstra’s algorithm proceeds by 
finding the length of a shortest path from a to a first vertex, the length of a shortest path from 
a to a second vertex, and so on, until the length of a shortest path from a to z is found. As a 
side benefit, this algorithm is easily extended to find the length of the shortest path from a to all 
other vertices of the graph, and not just to z. 

The algorithm relies on a series of iterations. A distinguished set of vertices is constructed 
by adding one vertex at each iteration. (This is the set S in Example 1.) A labeling procedure is 
carried out at each iteration. In this labeling procedure, a vertex w is labeled with the length of a 
shortest path from a to w that contains only vertices already in the distinguished set. The vertex 
added to the distinguished set is one with a minimal label among those vertices not already in 
the set. 

We now give the details of Dijkstra’s algorithm. It begins by labeling a with 0 and the 
other vertices with oo. We use the notation Ly(a) = 0 and Lp(v) = œ for these labels before 
any iterations have taken place (the subscript 0 stands for the “Oth” iteration). These labels are 
the lengths of shortest paths from a to the vertices, where the paths contain only the vertex a. 
(Because no path from a to a vertex different from a exists, co is the length of a shortest path 
between a and this vertex.) 

Dijkstra’s algorithm proceeds by forming a distinguished set of vertices. Let S, denote this 
set after k iterations of the labeling procedure. We begin with Sọ = Ø. The set S, is formed 
from S,_,; by adding a vertex u not in S,_, with the smallest label. 

Once u is added to S,, we update the labels of all vertices not in S,, so that L,(v), the label 
of the vertex v at the kth stage, is the length of a shortest path from a to v that contains vertices 
only in S, (that is, vertices that were already in the distinguished set together with u). Note that 
the way we choose the vertex u to add to S, at each step is an optimal choice at each step, making 


EDSGER WYBE DIJKSTRA (1930-2002) Edsger Dijkstra, born in the Netherlands, began programming 
computers in the early 1950s while studying theoretical physics at the University of Leiden. In 1952, realizing 
that he was more interested in programming than in physics, he quickly completed the requirements for his 
physics degree and began his career as a programmer, even though programming was not a recognized profes- 
sion. (In 1957, the authorities in Amsterdam refused to accept “programming” as his profession on his marriage 
license. However, they did accept “theoretical physicist” when he changed his entry to this.) 

Dijkstra was one of the most forceful proponents of programming as a scientific discipline. He has 
made fundamental contributions to the areas of operating systems, including deadlock avoidance; program- 
ming languages, including the notion of structured programming; and algorithms. In 1972 Dijkstra re- 
ceived the Turing Award from the Association for Computing Machinery, one of the most prestigious 


awards in computer science. Dijkstra became a Burroughs Research Fellow in 1973, and in 1984 he was appointed to a chair in 
Computer Science at the University of Texas, Austin. 
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this a greedy algorithm. (We will prove shortly that this greedy algorithm always produces an 
optimal solution.) 

Let v be a vertex not in S,. To update the label of v, note that L,(v) is the length of a shortest 
path from a to v containing only vertices in $,. The updating can be carried out efficiently when 
this observation is used: A shortest path from a to v containing only elements of S, is either a 
shortest path from a to v that contains only elements of S,_, (that is, the distinguished vertices 
not including u), or it is a shortest path from a to u at the (k — 1)st stage with the edge {u, v} 
added. In other words, 


La v) = min{L,_,(q, v), L(a, u) + w(u, v)}, 


where w(u, v) is the length of the edge with u and v as endpoints. This procedure is iterated by 
successively adding vertices to the distinguished set until z is added. When z is added to the 
distinguished set, its label is the length of a shortest path from a to z. 

Diykstra’s algorithm is given in Algorithm 1. Later we will give a proof that this algorithm 
is correct. Note that we can find the length of the shortest path from a to all other vertices of the 
graph if we continue this procedure until all vertices are added to the distinguished set. 


ALGORITHM 1 Dijkstra’s Algorithm. 


procedure Dijkstra(G: weighted connected simple graph, with 
all weights positive) 

{G has vertices a = vo, Vi, ..., V, = Z and lengths w(v; v;) 
where w(v,, v) = 00 if {v,, v} is not an edge in G} 

fori := 1 ton 
L(v;) := œ 

L(a) := 0 

S:=G 

{the labels are now initialized so that the label of a is 0 and all 
other labels are co, and S is the empty set} 


while z ¢ S 
u := a vertex not in S with L(u) minimal 
S:=Svu {u} 


for all vertices v not in S 
if L(u) + w(u, v) < L(v) then L(v) := L(u) + w(u, v) 
{this adds a vertex to S with minimal label and updates the 
labels of vertices not in S$} 
return L(z) {L(z) = length of a shortest path from a to z} 


Example 2 illustrates how Dijkstra’s algorithm works. Afterward, we will show that this 
algorithm always produces the length of a shortest path between two vertices in a weighted 
graph. 


Use Dijkstra’s algorithm to find the length of a shortest path between the vertices a and z in the 
weighted graph displayed in Figure 4(a). 


Solution: The steps used by Dijkstra’s algorithm to find a shortest path between a and z are shown 
in Figure 4. At each iteration of the algorithm the vertices of the set S, are circled. A shortest 
path from a to each vertex containing only vertices in S, is indicated for each iteration. The 
algorithm terminates when z is circled. We find that a shortest path from a to z is a, c, b, d, e, z, 
with length 13. < 
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FIGURE 4 Using Dijkstra’s algorithm to find a shortest path from a to z. 


Remark: In performing Dijkstra’s algorithm it is sometimes more convenient to keep track of 
labels of vertices in each step using a table instead of redrawing the graph for each step. 


Next, we use an inductive argument to show that Dijkstra’s algorithm produces the length 
of a shortest path between two vertices a and z in an undirected connected weighted graph. Take 
as the inductive hypothesis the following assertion: At the kth iteration 


(i) the label of every vertex v in S is the length of a shortest path from a to this vertex, and 


(ii) the label of every vertex not in S is the length of a shortest path from a to this vertex that 
contains only (besides the vertex itself) vertices in S. 


When k = 0, before any iterations are carried out, S = Ø, so the length of a shortest path from a 
to a vertex other than a is oo. Hence, the basis case is true. 

Assume that the inductive hypothesis holds for the kth iteration. Let v be the vertex added 

È to S at the (k + 1)st iteration, so v is a vertex not in S at the end of the kth iteration with the 
smallest label (in the case of ties, any vertex with smallest label may be used). 

From the inductive hypothesis we see that the vertices in S before the (k + 1)st iteration are 
labeled with the length of a shortest path from a. Also, v must be labeled with the length of a 
shortest path to it from a. If this were not the case, at the end of the kth iteration there would 
be a path of length less than L,(v) containing a vertex not in S [because L,(v) is the length of a 
shortest path from a to v containing only vertices in S after the kth iteration]. Let u be the first 
vertex not in S in such a path. There is a path with length less than L,(v) from a to u containing 
only vertices of S. This contradicts the choice of v. Hence, (i) holds at the end of the (k + 1)st 
iteration. 

Let u be a vertex not in S after k + 1 iterations. A shortest path from a to u containing only 
elements of S either contains v or it does not. If it does not contain v, then by the inductive 


THEOREM 1 


THEOREM 2 
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hypothesis its length is L,(u). If it does contain v, then it must be made up of a path from a 
to v of shortest possible length containing elements of S other than v, followed by the edge 
from v to u. In this case, its length would be L,(v) + wy, u). This shows that (ii) is true, because 
L100) = min{L,(u), L,(v) + wy, u)}. 

We now state the thereom that we have proved. 


Dykstra’s algorithm finds the length of a shortest path between two vertices in a connected 
simple undirected weighted graph. 


We can now estimate the computational complexity of Dijkstra’s algorithm (in terms of 
additions and comparisons). The algorithm uses no more than n — | iterations where n is the 
number of vertices in the graph, because one vertex is added to the distinguished set at each 
iteration. We are done if we can estimate the number of operations used for each iteration. We 
can identify the vertex not in S, with the smallest label using no more than n — 1 comparisons. 
Then we use an addition and a comparison to update the label of each vertex not in S,. It follows 
that no more than 2(n — 1) operations are used at each iteration, because there are no more 
than n — 1 labels to update at each iteration. Because we use no more than n — 1 iterations, each 
using no more than 2(n — 1) operations, we have Theorem 2. 


Dijkstra’s algorithm uses O(n?) operations (additions and comparisons) to find the length of 
a shortest path between two vertices in a connected simple undirected weighted graph with n 
vertices. 


10.6.3 The Traveling Salesperson Problem 


We now discuss an important problem involving weighted graphs. Consider the following prob- 
lem: A traveling salesperson wants to visit each of n cities exactly once and return to his starting 
point. For example, suppose that the salesperson wants to visit Detroit, Toledo, Saginaw, Grand 
Rapids, and Kalamazoo (see Figure 5). In which order should he visit these cities to travel the 
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FIGURE 5 The graph showing the distances between five cities. 
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An 1832 handbook Der 
Handlungsreisende (The 
Traveling Salesperson) 
mentions the traveling 
salesman problem, with 
sample tours through 
Germany and 
Switzerland. 


minimum total distance? To solve this problem we can assume the salesperson starts in Detroit 
(because this must be part of the circuit) and examine all possible ways for him to visit the other 
four cities and then return to Detroit (starting elsewhere will produce the same circuits). There 
are a total of 24 such circuits, but because we travel the same distance when we travel a circuit in 
reverse order, we need only consider 12 different circuits to find the minimum total distance he 
must travel. We list these 12 different circuits and the total distance traveled for each circuit. As 
can be seen from the list, the minimum total distance of 458 miles is traveled using the circuit 
Detroit-Toledo—Kalamazoo—Grand Rapids—Saginaw—Detroit (or its reverse). 


Total Distance (miles) 


Detroit-Toledo—Grand Rapids—Saginaw—Kalamazoo—Detroit 
Detroit-Toledo—Grand Rapids—Kalamazoo-Saginaw-Detroit 
Detroit-Toledo—Kalamazoo—Saginaw-Grand Rapids—Detroit 
Detroit-Toledo—Kalamazoo-Grand Rapids—Saginaw-Detroit 
Detroit-Toledo—Saginaw—Kalamazoo—Grand Rapids—Detroit 
Detroit-Toledo—-Saginaw—Grand Rapids—Kalamazoo-Detroit 
Detroit-Saginaw—Toledo—Grand Rapids—Kalamazoo-Detroit 
Detroit-Saginaw—Toledo—Kalamazoo—Grand Rapids—Detroit 
Detroit-Saginaw—Kalamazoo—Toledo—Grand Rapids—Detroit 
Detroit-Saginaw—Grand Rapids—Toledo—Kalamazoo-Detroit 
Detroit-Grand Rapids—Saginaw—Toledo—Kalamazoo-Detroit 
Detroit-Grand Rapids—Toledo—Saginaw—Kalamazoo-Detroit 


We just described an instance of the traveling salesperson problem. The traveling sales- 
person problem asks for the circuit of minimum total weight in a weighted, complete, undirected 
graph that visits each vertex exactly once and returns to its starting point. This is equivalent to 
asking for a Hamilton circuit with minimum total weight in the complete graph, because each 
vertex is visited exactly once in the circuit. 

The most straightforward way to solve an instance of the traveling salesperson problem is 
to examine all possible Hamilton circuits and select one of minimum total length. How many 
circuits do we have to examine to solve the problem if there are n vertices in the graph? Once a 
starting point is chosen, there are (n — 1)! different Hamilton circuits to examine, because there 
are n — | choices for the second vertex, n — 2 choices for the third vertex, and so on. Because a 
Hamilton circuit can be traveled in reverse order, we need only examine (n — 1)!/2 circuits to 
find our answer. Note that (n — 1)!/2 grows extremely rapidly. Trying to solve a traveling sales- 
person problem in this way when there are only a few dozen vertices is impractical. For example, 
with 25 vertices, a total of 24!/2 (approximately 3.1 x 107°) different Hamilton circuits would 
have to be considered. If it took just one nanosecond (107° second) to examine each Hamilton 
circuit, a total of approximately ten million years would be required to find a minimum-length 
Hamilton circuit in this graph by exhaustive search techniques. 

Because the traveling salesperson problem has both practical and theoretical importance, a 
great deal of effort has been devoted to devising efficient algorithms that solve it. However, no 
algorithm with polynomial worst-case time complexity is known for solving this problem. Fur- 
thermore, if a polynomial worst-case time complexity algorithm were discovered for the travel- 
ing salesperson problem, many other difficult problems would also be solvable using polynomial 
worst-case time complexity algorithms (such as determining whether a proposition in n vari- 
ables is a tautology, discussed in Chapter 1). This follows from the theory of NP-completeness. 
(For more information about this, consult [GaJo79].) 

A practical approach to the traveling salesperson problem when there are many vertices 
to visit is to use an approximation algorithm. These are algorithms that do not necessarily 
produce the exact solution to the problem but instead are guaranteed to produce a solution that is 
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close to an exact solution. (Also, see the preamble to Exercise 46 in the Supplmentary Exercises 
of Chapter 3.) That is, they may produce a Hamilton circuit with total weight W’ such that 
W < W’ < cW, where W is the total length of an exact solution and c is a constant. For example, 
there is an algorithm with polynomial worst-case time complexity that works if the weighted 
graph satisfies the triangle inequality such that c = 3/2. For general weighted graphs for every 
positive real number k no algorithm is known that will always produce a solution at most k times 
a best solution. If such an algorithm existed, this would show that the class P would be the same 
as the class NP, perhaps the most famous open question about the complexity of algorithms (see 
Section 3.3). 

In practice, algorithms have been developed that can solve traveling salesperson problems 
with as many as 1000 vertices within 2% of an exact solution using only a few minutes of com- 
puter time. For more information about the traveling salesperson problem, including history, 
applications, and algorithms, see the chapter on this topic in Applications of Discrete Mathe- 
matics [MiRo91] also available on the website for this book. 


1. For each of these problems about a subway system, de- 6. Find the length of a shortest path between these pairs of 


scribe a weighted graph model that can be used to solve 


vertices in the weighted graph in Exercise 3. 


the problem. a) aandd 
a) What is the least amount of time required to travel b) aandf 
between two stops? c) candf 

d) band z 


b) What is the minimum distance that can be traveled to 


reach a stop from another stop? 7. Find shortest paths in the weighted graph in Exercise 3 
c) What is the least fare required to travel between two between the pairs of weruges in Exercise 6. 
stops if fares between stops are added to give the total 8. Find a shortest path (in mileage) between each of the 


fare? 


In Exercises 2—4 find the length of a shortest path between a 
and z in the given weighted graph. 


2. 


5. Find a shortest path between a and z in each of the 
weighted graphs in Exercises 2—4. 


following pairs of cities in the airline system shown in 
Figure 1. 

a) New York and Los Angeles 

b) Boston and San Francisco 

c) Miami and Denver 

d) Miami and Los Angeles 


5 d 


9. Find a combination of flights with the least total air time 
between the pairs of cities in Exercise 8, using the flight 
times shown in Figure 1. 

10. Find a least expensive combination of flights connecting 
the pairs of cities in Exercise 8, using the fares shown in 
Figure 1. 

11. Finda shortest route (in distance) between computer cen- 


ters in each of these pairs of cities in the communications 
network shown in Figure 2. 


a) Boston and Los Angeles 
b) New York and San Francisco 


c) Dallas and San Francisco 
d) Denver and New York 


12. Find a route with the shortest response time between the 
pairs of computer centers in Exercise 11 using the re- 
sponse times given in Figure 2. 

13. Find a least expensive route, in monthly lease charges, 
between the pairs of computer centers in Exercise 11 us- 
ing the lease charges given in Figure 2. 

14. Explain how to find a path with the least number of edges 
between two vertices in an undirected graph by consider- 
ing it as a shortest path problem in a weighted graph. 
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15. Extend Dijkstra’s algorithm for finding the length of a 
shortest path between two vertices in a weighted simple 
connected graph so that the length of a shortest path be- 
tween the vertex a and every other vertex of the graph is 
found. 


16. Extend Dijkstra’s algorithm for finding the length of a 
shortest path between two vertices in a weighted simple 
connected graph so that a shortest path between these ver- 
tices is constructed. 


17. The weighted graphs in the figures here show some major 
roads in New Jersey. Part (a) shows the distances between 
cities on these roads; part (b) shows the tolls. 


Newark 


42 Woodbridge 
Se! 3 


(b) 


a) Finda shortest route in distance between Newark and 
Camden, and between Newark and Cape May, using 
these roads. 


b) Find a least expensive route in terms of total tolls us- 
ing the roads in the graph between the pairs of cities 
in part (a) of this exercise. 


18. Is a shortest path between two vertices in a weighted 
graph unique if the weights of edges are distinct? 


19. What are some applications where it is necessary to find 
the length of a longest simple path between two vertices 
in a weighted graph? 


20. What is the length of a longest simple path in the 
weighted graph in Figure 4 between a and z? Between 
c and z? 


Floyd’s algorithm, displayed as Algorithm 2, can be used to 
find the length of a shortest path between all pairs of vertices 
in a weighted connected simple graph. However, this algo- 
rithm cannot be used to construct shortest paths. (We assign 
an infinite weight to any pair of vertices not connected by an 
edge in the graph.) 


21. Use Floyd’s algorithm to find the distance between all 
pairs of vertices in the weighted graph in Figure 4(a). 


*22. Prove that Floyd’s algorithm determines the shortest dis- 
tance between all pairs of vertices in a weighted simple 
graph. 


*23. Give a big-O estimate of the number of operations (com- 
parisons and additions) used by Floyd’s algorithm to de- 
termine the shortest distance between every pair of ver- 
tices in a weighted simple graph with n vertices. 


*24. Show that Dijkstra’s algorithm may not work if edges can 
have negative weights. 


ALGORITHM 2 Floyd’s Algorithm. 


procedure Floyd(G: weighted simple graph) 
{G has vertices v}, vz, ..., V, and weights w(v,, v;) 
with w(v;, v) =% if {v,, vi} is not an edge} 
for i := 1 ton 
forj := 1 ton 
dv; v;) = wv, v;) 
for i := 1 ton 
for j:=1ton 
fork :=1ton 
if d(v;, vj) + dp vg) < Ay; Vy) 
then d(v;, v) := d(v;, v;i) + dv; Vy) 
return [dv,, v)| {d(v;, v;) is the length of a shortest 
path between v; and v, for 1 <i<n,1<j<n} 


25. Solve the traveling salesperson problem for this graph by 
finding the total weight of all Hamilton circuits and de- 
termining a circuit with minimum total weight. 


26. Solve the traveling salesperson problem for this graph by 
finding the total weight of all Hamilton circuits and de- 
termining a circuit with minimum total weight. 


27. Find a route with the least total airfare that visits each of 
the cities in this graph, where the weight on an edge is the 
least price available for a flight between the two cities. 


Detroit 


New 
York 


Los Angeles Denver 


28. Find a route with the least total airfare that visits each of 
the cities in this graph, where the weight on an edge is the 
least price available for a flight between the two cities. 
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29. Construct a weighted undirected graph such that the total 
weight of a circuit that visits every vertex at least once 
is minimized for a circuit that visits some vertices more 
than once. [Hint: There are examples with three vertices. ] 

30. Show that the problem of finding a circuit of minimum 
total weight that visits every vertex of a weighted graph 
at least once can be reduced to the problem of finding a 
circuit of minimum total weight that visits each vertex 
of a weighted graph exactly once. Do so by constructing 
a new weighted graph with the same vertices and edges 
as the original graph but whose weight of the edge con- 
necting the vertices u and v is equal to the minimum total 
weight of a path from u to v in the original graph. 


* 31. The longest path problem in a weighted directed graph 
with no simple circuits asks for a path in this graph such 
that the sum of its edge weights is a maximum. De- 
vise an algorithm for solving the longest path problem. 
[Hint: First find a topological ordering of the vertices of 
the graph. ] 


10.7.1 Introduction 


Consider the problem of joining three houses to each of three separate utilities, as shown in 

Linke Figure 1. Is it possible to join these houses and utilities so that none of the connections cross? 
This problem can be modeled using the complete bipartite graph K, ,. The original question can 
be rephrased as: Can K, ; be drawn in the plane so that no two of its edges cross? 


FIGURE 1 Three houses and three utilities. 
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EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


X< 


FIGURE 2 The 
graph K4. 


In this section we will study the question of whether a graph can be drawn in the plane 
without edges crossing. In particular, we will answer the houses-and-utilities problem. 

There are always many ways to represent a graph. When is it possible to find at least one 
way to represent this graph in a plane without any edges crossing? 


A graph is called planar if it can be drawn in the plane without any edges crossing (where 
a crossing of edges is the intersection of the lines or arcs representing them at a point other 
than their common endpoint). Such a drawing is called a planar representation of the graph. 


A graph may be planar even if it is usually drawn with crossings, because it may be possible 
to draw it in a different way without crossings. 


Is K, (shown in Figure 2 with two edges crossing) planar? 


Solution: K, is planar because it can be drawn without crossings, as shown in Figure 3. < 


Is Q}, shown in Figure 4, planar? 


Solution: Q} is planar, because it can be drawn without any edges crossing, as shown in 
Figure 5. < 


We can show that a graph is planar by displaying a planar representation. It is harder to 
show that a graph is nonplanar. We will give an example to show how this can be done in an ad 
hoc fashion. Later we will develop some general results that can be used to do this. 


Is K33, shown in Figure 6, planar? 


Solution: Any attempt to draw K; 3 in the plane with no edges crossing is doomed. We now 
show why. In any planar representation of K} 3, the vertices v; and v, must be connected to both 
v4 and v;. These four edges form a closed curve that splits the plane into two regions, R, and 
R,, as shown in Figure 7(a). The vertex v3 is in either R; or Ry. When v; is in R}, the inside 
of the closed curve, the edges between v} and v4 and between v3 and v; separate R, into two 
subregions, R); and R,,, as shown in Figure 7(b). 

Next, note that there is no way to place the final vertex vg without forcing a crossing. For 
if vg is in R}, then the edge between vę and v} cannot be drawn without a crossing. If v¢ is in 
R,,, then the edge between v, and v¢ cannot be drawn without a crossing. If ve is in R», then 
the edge between v, and v¢ cannot be drawn without a crossing. 


Noe 


FIGURE 3 K, drawn FIGURE 4 The FIGURE 5 A planar 
with no crossings. graph Q}. representation of Q}. 
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vy V5 Vy V5 
vy v2 V3 
R2 Ry Ry 
V4 v2 V4 v2 
Ya v3 v6 (a) (b) 
FIGURE 6 The graph K;,. FIGURE 7 Showing that K,, is nonplanar. 


A similar argument can be used when v, is in R,. The completion of this argument is left 
for the reader (see Exercise 10). It follows that K, 3 is not planar. < 


Example 3 solves the utilities-and-houses problem that was described at the beginning of 
this section. The three houses and three utilities cannot be connected in the plane without a 
crossing. A similar argument can be used to show that K, is nonplanar. (See Exercise 11.) 


APPLICATIONS OF PLANAR GRAPHS Planarity of graphs plays an important role in the 
design of electronic circuits. We can model a circuit with a graph by representing components 
of the circuit by vertices and connections between them by edges. We can print a circuit on a 
single board with no connections crossing if the graph representing the circuit is planar. When 
this graph is not planar, we must turn to more expensive options. For example, we can partition 
the vertices in the graph representing the circuit into planar subgraphs. We then construct the 
circuit using multiple layers. (See the preamble to Exercise 30 to learn about the thickness of a 
graph.) We can construct the circuit using insulated wires whenever connections cross. In this 
case, drawing the graph with the fewest possible crossings is important. (See the preamble to 
Exercise 26 to learn about the crossing number of a graph.) 

The planarity of graphs is also useful in the design of road networks. Suppose we want 
to connect a group of cities by roads. We can model a road network connecting these cities 
using a simple graph with vertices representing the cities and edges representing the highways 
connecting them. We can built this road network without using underpasses or overpasses if the 
resulting graph is planar. 


10.7.2 Euler’s Formula 


A planar representation of a graph splits the plane into regions, including an unbounded region. 
For instance, the planar representation of the graph shown in Figure 8 splits the plane into six 
regions. These are labeled in the figure. Euler showed that all planar representations of a graph 
split the plane into the same number of regions. He accomplished this by finding a relationship 
among the number of regions, the number of vertices, and the number of edges of a planar 
graph. 


FIGURE 8 The regions of the planar representation of a graph. 
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FIGURE 9 The 
basis case of the 
proof of Euler’s 
formula. 


EXAMPLE 4 


EULER’S FORMULA Let G be a connected planar simple graph with e edges and v 
vertices. Let r be the number of regions in a planar representation of G. Then r = e — v +2. 


Proof: First, we specify a planar representation of G. We will prove the theorem by constructing 
a sequence of subgraphs G,, G>, ..., G, = G, successively adding an edge at each stage. This is 
done using the following inductive definition. Arbitrarily pick one edge of G to obtain G. Obtain 
G, from G,_, by arbitrarily adding an edge that is incident with a vertex already in G,_,, adding 
the other vertex incident with this edge if it is not already in G,_,. This construction is possible 
because G is connected. G is obtained after e edges are added. Let r,, e,, and v, represent the 
number of regions, edges, and vertices of the planar representation of G, induced by the planar 
representation of G, respectively. 

The proof will now proceed by induction. The relationship r} = e} — v; + 2 is true for G}, 
because e, = 1, v; = 2, and r; = 1. This is shown in Figure 9. 

Now assume that r, = e, — vy + 2. Let {a,,), bg41 } be the edge that is added to G, to obtain 
G,,,. There are two possibilities to consider. In the first case, both a,,,; and b,,, are already 
in G,. These two vertices must be on the boundary of a common region R, or else it would be 
impossible to add the edge {a,,,, 5;,, } to G, without two edges crossing (and G,,, is planar). 
The addition of this new edge splits R into two regions. Consequently, in this case, r}; =r, + 
l, &4; =e, + 1, and v,,,; = v}. Thus, each side of the formula relating the number of regions, 
edges, and vertices increases by exactly one, so this formula is still true. In other words, r}; = 
ek41 — Veg, + 2. This case is illustrated in Figure 10(a). 

In the second case, one of the two vertices of the new edge is not already in G,. Suppose that 
y+) isin G but that b,, ; is not. Adding this new edge does not produce any new regions, because 
bı must be in a region that has a,,,; on its boundary. Consequently, r,,; = r}. Moreover, e,,, = 
e, + l and v,,, =v, + 1. Each side of the formula relating the number of regions, edges, and 
vertices remains the same, so the formula is still true. In other words, r4) = e41 — Vg] +2. 
This case is illustrated in Figure 10(b). 

We have completed the induction argument. Hence, r, = e, —v, +2 for all n. Because 
the original graph is the graph G,, obtained after e edges have been added, the theorem is 
true. < 


Euler’s formula is illustrated in Example 4. 


Suppose that a connected planar simple graph has 20 vertices, each of degree 3. Into how many 
regions does a representation of this planar graph split the plane? 


an+1 


bn +1 
(a) (b) 


FIGURE 10 Adding an edge to G, to produce G,,, ;. 


COROLLARY 1 


COROLLARY 2 
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FIGURE 11 The degrees of regions. 


Solution: This graph has 20 vertices, each of degree 3, so v = 20. Because the sum of the degrees 
of the vertices, 3v = 3 - 20 = 60, is equal to twice the number of edges, 2e, we have 2e = 60, or 
e = 30. Consequently, from Euler’s formula, the number of regions is 


r=e-v+2=30-2042= 12. < 


Euler’s formula can be used to establish some inequalities that must be satisfied by planar 
graphs. One such inequality is given in Corollary 1. 


If G is a connected planar simple graph with e edges and v vertices, where v > 3, then e < 
3v — 6. 


Before we prove Corollary 1 we will use it to prove the following useful result. 


If G is a connected planar simple graph, then G has a vertex of degree not exceeding five. 


Proof: If G has one or two vertices, the result is true. If G has at least three vertices, by Corol- 
lary 1 we know that e < 3v — 6, so 2e < 6v — 12. If the degree of every vertex were at least six, 
then because 2e = per deg(v) (by the handshaking theorem), we would have 2e > 6v. But this 
contradicts the inequality 2e < 6v — 12. It follows that there must be a vertex with degree no 
greater than five. < 


The proof of Corollary 1 is based on the concept of the degree of a region, which is defined 
to be the number of edges on the boundary of this region. When an edge occurs twice on the 
boundary (so that it is traced out twice when the boundary is traced out), it contributes two to 
the degree. We denote the degree of a region R by deg(R). The degrees of the regions of the 
graph shown in Figure 11 are displayed in the figure. 

The proof of Corollary 1 can now be given. 


Proof: A connected planar simple graph drawn in the plane divides the plane into regions, 
say r of them. The degree of each region is at least three. (Because the graphs discussed here 
are simple graphs, no multiple edges that could produce regions of degree two, or loops that 
could produce regions of degree one, are permitted.) In particular, note that the degree of the 
unbounded region is at least three because there are at least three vertices in the graph. 
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Note that the sum of the degrees of the regions is exactly twice the number of edges in 
the graph, because each edge occurs on the boundary of a region exactly twice (either in two 
different regions, or twice in the same region). Because each region has degree greater than or 
equal to three, it follows that 


2e= » deg(R) > 3r. 


all regions R 
Hence, 
(2/3)e 2r. 
Using r = e — v + 2 (Euler’s formula), we obtain 
e—v+2 < (2/3)e. 
It follows that e/3 < v — 2. This shows that e < 3v — 6. < 


This corollary can be used to demonstrate that K; is nonplanar. 


Show that Ķ; is nonplanar using Corollary 1. 


Solution: The graph K; has five vertices and 10 edges. However, the inequality e < 3v — 6 is 
not satisfied for this graph because e = 10 and 3v — 6 = 9. Therefore, K; is not planar. < 


It was previously shown that K, ; is not planar. Note, however, that this graph has six vertices 
and nine edges. This means that the inequality e = 9 < 12 = 3 - 6 — 6 is satisfied. Consequently, 
the fact that the inequality e < 3v — 6 is satisfied does not imply that a graph is planar. However, 
the following corollary of Theorem 1 can be used to show that K, 3 is nonplanar. 


If a connected planar simple graph has e edges and v vertices with v > 3 and no circuits of 
length three, then e < 2v — 4. 


The proof of Corollary 3 is similar to that of Corollary 1, except that in this case the fact that 
there are no circuits of length three implies that the degree of a region must be at least four. The 
details of this proof are left for the reader (see Exercise 15). 


Use Corollary 3 to show that K, ; is nonplanar. 


Solution: Because K3 has no circuits of length three (this is easy to see because it is bipartite), 
Corollary 3 can be used. K; has six vertices and nine edges. Because e = 9 and 2v — 4 = 8, 
Corollary 3 shows that K; 3 is nonplanar. < 


10.7.3 Kuratowski’s Theorem 


We have seen that K; 3 and K; are not planar. Clearly, a graph is not planar if it contains either 
of these two graphs as a subgraph. Surprisingly, all nonplanar graphs must contain a subgraph 
that can be obtained from K, or K; using certain permitted operations. 

If a graph is planar, so will be any graph obtained by removing an edge {u, v} and adding a 
new vertex w together with edges {u, w} and {w, v}. Such an operation is called an elementary 
subdivision. The graphs G, = (V,, E,) and G, = (V3, E,) are called homeomorphic if they can 
be obtained from the same graph by a sequence of elementary subdivisions. 


EXAMPLE 7 


THEOREM 2 


EXAMPLE 8 


Extra » 
Examples 
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Photo 
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FIGURE 12 Homeomorphic graphs. 


Show that the graphs G,, G,, and G, displayed in Figure 12 are all homeomorphic. 


Solution: These three graphs are homeomorphic because all three can be obtained from G, by 
elementary subdivisions. G, can be obtained from itself by an empty sequence of elementary 
subdivisions. To obtain G, from G} we can use this sequence of elementary subdivisions: (i) re- 
move the edge {a, c}, add the vertex f, and add the edges {a, f } and { f, c}; (ii) remove the edge 
{b, c}, add the vertex g, and add the edges {b, g} and {g,c}; and (iii ) remove the edge {b, g}, 
add the vertex h, and add the edges {g, h} and {b, h}. We leave it to the reader to determine the 
sequence of elementary subdivisions needed to obtain G, from G}. < 


The Polish mathematician Kazimierz Kuratowski established Theorem 2 in 1930, which 
characterizes planar graphs using the concept of graph homeomorphism. 


A graph is nonplanar if and only if it contains a subgraph homeomorphic to K} 3 or Ks. 


It is clear that a graph containing a subgraph homeomorphic to K; 3 or K; is nonplanar. However, 
the proof of the converse, namely that every nonplanar graph contains a subgraph homeomor- 
phic to K33 or K;, is complicated and will not be given here. Examples 8 and 9 illustrate how 
Kuratowski’s theorem is used. 


Determine whether the graph G shown in Figure 13 is planar. 


Solution: G has a subgraph H homeomorphic to K5. H is obtained by deleting A, j, and k and all 
edges incident with these vertices. H is homeomorphic to K; because it can be obtained from 
K; (with vertices a, b, c, g, and i) by a sequence of elementary subdivisions, adding the vertices 
d, e, and f. (The reader should construct such a sequence of elementary subdivisions.) Hence, 
G is nonplanar. < 


KAZIMIERZ KURATOWSKI (1896-1980) Kazimierz Kuratowski, the son of a famous Warsaw lawyer, 
attended secondary school in Warsaw. He studied in Glasgow, Scotland, from 1913 to 1914 but could not return 
there after the outbreak of World War I. In 1915 he entered Warsaw University, where he was active in the 
Polish patriotic student movement. He published his first paper in 1919 and received his Ph.D. in 1921. He 
was an active member of the group known as the Warsaw School of Mathematics, working in the areas of 
the foundations of set theory and topology. He was appointed associate professor at the Lwów Polytechnical 
University, where he stayed for seven years, collaborating with the important Polish mathematicians Banach 
and Ulam. In 1930, while at Lwów, Kuratowski completed his work characterizing planar graphs. 

In 1934 he returned to Warsaw University as a full professor. Until the start of World War II, he was 
active in research and teaching. During the war, because of the persecution of educated Poles, Kuratowski 
went into hiding under an assumed name and taught at the clandestine Warsaw University. After the war 


he helped revive Polish mathematics, serving as director of the Polish National Mathematics Institute. He wrote over 180 papers 
and three widely used textbooks. 
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FIGURE 13 The undirected graph G, a subgraph H homeomorphic to K;, and K;. 
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FIGURE 14 (a) The Petersen graph, (b) a subgraph H homeomorphic to K, ,, and (c) K; 3. 


EXAMPLE 9 _Isthe Petersen graph, shown in Figure 14(a), planar? (The Danish mathematician Julius Petersen 
studied this graph in 1891; it is often used to illustrate various theoretical properties of graphs.) 


Solution: The subgraph H of the Petersen graph obtained by deleting b and the three edges that 
have b as an endpoint, shown in Figure 14(b), is homeomorphic to K, 3, with vertex sets { f, d, j} 
and {e,i,h}, because it can be obtained by a sequence of elementary subdivisions, deleting 
{d, h} and adding {c,h} and {c, d}, deleting {e, f} and adding {a, e} and {a, f}, and deleting 


{i j} and adding {g, i} and {g, j}. Hence, the Petersen graph is not planar. < 
Exercises 
1. Can five houses be connected to two utilities without con- In Exercises 5—9 determine whether the given graph is planar. 
nections crossing? If so, draw it so that no edges cross. 
In Exercises 2—4 draw the given planar graph without any 5, á 6. a b 
crossings. 
2. 3. 
b c 
7 z J d e 
4. 7 a b 8. a b 
h ic 
f c 
g d 


10. 
11. 


12. 


13. 


14. 


15. 
16. 


*17. 


18. 


19. 


Complete the argument in Example 3. 


Show that K; is nonplanar using an argument similar to 
that given in Example 3. 


Suppose that a connected planar graph has eight vertices, 
each of degree three. Into how many regions is the plane 
divided by a planar representation of this graph? 


Suppose that a connected planar graph has six vertices, 
each of degree four. Into how many regions is the plane 
divided by a planar representation of this graph? 


Suppose that a connected planar graph has 30 edges. If a 
planar representation of this graph divides the plane into 
20 regions, how many vertices does this graph have? 


Prove Corollary 3. 


Suppose that a connected bipartite planar simple graph 
has e edges and v vertices. Show that e < 2v — 4 if v > 3. 


Suppose that a connected planar simple graph with e 
edges and v vertices contains no simple circuits of length 
4 or less. Show that e < (5/3)v — (10/3) if v > 4. 


Suppose that a planar graph has k connected components, 
e edges, and v vertices. Also suppose that the plane is 
divided into r regions by a planar representation of the 
graph. Find a formula for r in terms of e, v, and k. 


Which of these nonplanar graphs have the property that 
the removal of any vertex and all edges incident with that 
vertex produces a planar graph? 


a) K bK ©9K; @) Ky, 


In Exercises 20-22 determine whether the given graph is 
homeomorphic to K; 3. 


20. 


21. 


a b c 
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22. 


a 
l b 
k c 
j d 
i e 
h f 
8 


In Exercises 23—25 use Kuratowski’s theorem to determine 
whether the given graph is planar. 


23. a b c d 


25. 


The crossing number of a simple graph is the minimum 
number of crossings that can occur when this graph is drawn 
in the plane where no three arcs representing edges are per- 
mitted to cross at the same point. 


26. Show that K;; has 1 as its crossing number. 


** 27. Find the crossing numbers of each of these nonplanar 


graphs. 
a) K; b) Kg c) K; 
d) K34 e) Ky4 f) K55 


*28. Find the crossing number of the Petersen graph. 
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** 29, Show that if m and n are even positive integers, the cross- 32. Show that if G is a connected simple graph with v ver- 
ing number of K,,,,, is less than or equal to mn(m — 2) tices and e edges, where v > 3, then the thickness of G is 
(n — 2)/16. [Hint: Place m vertices along the x-axis so at least [e/(3v — 6)]. 
that they are equally spaced and symmetric about the ori- * 33. Use Exercise 32 to show that the thickness of K, is at 
gin and place n vertices along the y-axis so that they are least |(n + 7)/6| whenever n is a positive integer. 
equally spaced and symmetric about the origin. Now con- 34. Show that if G is a connected simple graph with v ver- 


nect each of the m vertices on the x-axis to each of the tices and e edges, where v > 3, and no circuits of length 


VErHEeS OT tHE Y AXIS anti coun Mie GrOsEm Esi] three, then the thickness of G is at least [e/(2v — 4)]. 
The thickness of a simple graph G is the smallest number of 35. Use Exercise 34 to show that the thickness of K, „, where 
planar subgraphs of G that have G as their union. m and n are not both 1, is at least [mn/(2m + 2n — 4)] 


whenever m and n are positive integers. 


30. Show that K3 has 2 as its thickness. * 36. Draw K; on the surface of a torus (a doughnut-shaped 
solid) so that no edges cross. 


* 31. Find the thickness of the graphs in Exercise 27. * 37. Draw K; on the surface of a torus so that no edges cross. 


Graph Coloring 


10.8.1 Introduction 


tink » Problems related to the coloring of maps of regions, such as maps of parts of the world, have 
generated many results in graph theory. When a map* is colored, two regions with a common 
border are customarily assigned different colors. One way to ensure that two adjacent regions 
never have the same color is to use a different color for each region. However, this is inefficient, 
and on maps with many regions it would be hard to distinguish similar colors. Instead, a small 
number of colors should be used whenever possible. Consider the problem of determining the 
least number of colors that can be used to color a map so that adjacent regions never have the 
same color. For instance, for the map shown on the left in Figure 1, four colors suffice, but 
three colors are not enough. (The reader should check this.) In the map on the right in Figure 1, 
three colors are sufficient (but two are not). 


PA AT 


FIGURE 1 Two maps. 


Each map in the plane can be represented by a graph. To set up this correspondence, each 
region of the map is represented by a vertex. Edges connect two vertices if the regions repre- 
sented by these vertices have a common border. Two regions that touch at only one point are 
not considered adjacent. The resulting graph is called the dual graph of the map. By the way 
in which dual graphs of maps are constructed, it is clear that any map in the plane has a planar 
dual graph. Figure 2 displays the dual graphs that correspond to the maps shown in Figure 1. 

The problem of coloring the regions of a map is equivalent to the problem of coloring the 
vertices of the dual graph so that no two adjacent vertices in this graph have the same color. We 
now define a graph coloring. 


*We will assume that all regions in a map are connected. This eliminates any problems presented by such geographical entities 
as Michigan. 


Definition 1 


Definition 2 


THEOREM 1 
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FIGURE 2_ Dual graphs of the maps in Figure 1. 


A coloring of a simple graph is the assignment of a color to each vertex of the graph so that 
no two adjacent vertices are assigned the same color. 


A graph can be colored by assigning a different color to each of its vertices. However, for most 
graphs a coloring can be found that uses fewer colors than the number of vertices in the graph. 
What is the least number of colors necessary? 


The chromatic number of a graph is the least number of colors needed for a coloring of 
this graph. The chromatic number of a graph G is denoted by y(G). (Here y is the Greek 
letter chi.) 


Note that asking for the chromatic number of a planar graph is the same as asking for the min- 
imum number of colors required to color a planar map so that no two adjacent regions are 
assigned the same color. This question has been studied for more than 100 years. The answer is 
provided by one of the most famous theorems in mathematics. 


THE FOUR COLOR THEOREM The chromatic number of a planar graph is no greater 
than four. 


The four color theorem was originally posed as a conjecture in the 1850s. It was finally 
proved by the American mathematicians Kenneth Appel and Wolfgang Haken in 1976. Prior to 
1976, many incorrect proofs were published, often with hard-to-find errors. In addition, many 
futile attempts were made to construct counterexamples by drawing maps that require more than 
four colors. (Proving the five color theorem is not that difficult; see Exercise 36.) 

Perhaps the most notorious fallacious proof in all of mathematics is the incorrect proof of 
the four color theorem published in 1879 by a London barrister and amateur mathematician, 
Alfred Kempe. Mathematicians accepted his proof as correct until 1890, when Percy Heawood 
found an error that made Kempe’s argument incomplete. However, Kempe’s line of reasoning 
turned out to be the basis of the successful proof given by Appel and Haken. Their proof relies on 
a careful case-by-case analysis carried out by computer. They showed that if the four color theo- 
rem were false, there would have to be a counterexample of one of approximately 2000 different 


| ALFRED BRAY KEMPE (1849-1922) Kempe was a barrister and a leading authority on ecclesiastical law. 


However, having studied mathematics at Cambridge University, he retained his interest in it, and later in life 
he devoted considerable time to mathematical research. Kempe made contributions to kinematics, the branch 
of mathematics dealing with motion, and to mathematical logic. However, Kempe is best remembered for his 
fallacious proof of the four color theorem. 
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FIGURE 3 The simple graphs G and H. 


types, and they then showed that none of these types exists. They used over 1000 hours of com- 
puter time in their proof. This proof generated a large amount of controversy, because computers 
played such an important role in it. For example, could there be an error in a computer program 
that led to incorrect results? Was their argument really a proof if it depended on what could be 
unreliable computer output? Since their proof appeared, simpler proofs that rely on checking 
fewer types of possible counterexamples have been found and a proof using an automated proof 
system has been created. However, no proof not relying on a computer has yet been found. 

Note that the four color theorem applies only to planar graphs. Nonplanar graphs can have 
arbitrarily large chromatic numbers, as will be shown in Example 2. 

Two things are required to show that the chromatic number of a graph is k. First, we must 
show that the graph can be colored with k colors. This can be done by constructing such a 
coloring. Second, we must show that the graph cannot be colored using fewer than k colors. 
Examples 1—4 illustrate how chromatic numbers can be found. 


What are the chromatic numbers of the graphs G and H shown in Figure 3? 


Solution: The chromatic number of G is at least three, because the vertices a, b, and c must be 
assigned different colors. To see if G can be colored with three colors, assign red to a, blue 
to b, and green to c. Then, d can (and must) be colored red because it is adjacent to b and c. 
Furthermore, e can (and must) be colored green because it is adjacent only to vertices colored 
red and blue, and f can (and must) be colored blue because it is adjacent only to vertices colored 
red and green. Finally, g can (and must) be colored red because it is adjacent only to vertices 
colored blue and green. This produces a coloring of G using exactly three colors. Figure 4 
displays such a coloring. 

The graph H is made up of the graph G with an edge connecting a and g. Any attempt to 
color H using three colors must follow the same reasoning as that used to color G, except at the 
last stage, when all vertices other than g have been colored. Then, because g is adjacent (in H) 
to vertices colored red, blue, and green, a fourth color, say brown, needs to be used. Hence, H 
has a chromatic number equal to 4. A coloring of H is shown in Figure 4. < 


b Blue e Green 
g Red a 


Blue Green e 
> Red > 


c Green fBlue c Green fBlue 
G H 


S 


Brown 


FIGURE 4 _ Colorings of the graphs G and H. 
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a Red b Blue 
a Red b Red c Red 
Brown e c Green 
d Yellow d Blue e Blue f Blue g Blue 
FIGURES A coloring of K;. FIGURE 6 A coloring of K, 4. 


What is the chromatic number of K,,? 


Solution: A coloring of K, can be constructed using n colors by assigning a different color 
to each vertex. Is there a coloring using fewer colors? The answer is no. No two vertices can 
be assigned the same color, because every two vertices of this graph are adjacent. Hence, the 
chromatic number of K, is n. That is, y(K,,) = n. (Recall that K, is not planar when n > 5, so 
this result does not contradict the four color theorem.) A coloring of K; using five colors is 
shown in Figure 5. < 


What is the chromatic number of the complete bipartite graph K,,, „, where m and n are positive 


integers? 


n,n? 


Solution: The number of colors needed may seem to depend on m and n. However, as Theorem 4 
in Section 10.2 tells us, only two colors are needed, because K,,,,, is a bipartite graph. Hence, 
X(K,y,,) = 2. This means that we can color the set of m vertices with one color and the set of 
n vertices with a second color. Because edges connect only a vertex from the set of m vertices 
and a vertex from the set of n vertices, no two adjacent vertices have the same color. A coloring 


of K3, with two colors is displayed in Figure 6. < 


What is the chromatic number of the graph C,,, where n > 3? (Recall that C, is the cycle with 


n? 
n vertices.) 


Solution: We will first consider some individual cases. To begin, let n = 6. Pick a vertex and 
color it red. Proceed clockwise in the planar depiction of C, shown in Figure 7. It is necessary to 
assign a second color, say blue, to the next vertex reached. Continue in the clockwise direction; 


HISTORICAL NOTE In 1852, an ex-student of Augustus De Morgan, Francis Guthrie, noticed that the coun- 
ties in England could be colored using four colors so that no adjacent counties were assigned the same color. 
On this evidence, he conjectured that the four color theorem was true. Francis told his brother Frederick, at 
that time a student of De Morgan, about this problem. Frederick in turn asked his teacher De Morgan about 
his brother’s conjecture. De Morgan was extremely interested in this problem and publicized it throughout the 
mathematical community. In fact, the first written reference to the conjecture can be found in a letter from De 
Morgan to Sir William Rowan Hamilton. Although De Morgan thought Hamilton would be interested in this 
problem, Hamilton apparently was not interested in it, because it had nothing to do with quaternions. 


HISTORICAL NOTE Although a simpler proof of the four color theorem was found by Robertson, Sanders, 
Seymour, and Thomas in 1996, reducing the computational part of the proof to examining 633 configurations, 
no proof that does not rely on extensive computation has yet been found. 
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Links > 


EXAMPLE 5 


FIGURE 7 Colorings of C; and C,. 


the third vertex can be colored red, the fourth vertex blue, and the fifth vertex red. Finally, the 
sixth vertex, which is adjacent to the first, can be colored blue. Hence, the chromatic number of 
C; is 2. Figure 7 displays the coloring constructed here. 

Next, let n = 5 and consider C;. Pick a vertex and color it red. Proceeding clockwise, it 
is necessary to assign a second color, say blue, to the next vertex reached. Continuing in the 
clockwise direction, the third vertex can be colored red, and the fourth vertex can be colored 
blue. The fifth vertex cannot be colored either red or blue, because it is adjacent to the fourth 
vertex and the first vertex. Consequently, a third color is required for this vertex. Thus, the 
chromatic number of C; is 3. A coloring of C; using three colors is displayed in Figure 7. 

In general, two colors are needed to color C,, when n is even. To construct such a coloring, 
simply pick a vertex and color it red. Proceed around the graph in a clockwise direction (us- 
ing a planar representation of the graph) coloring the second vertex blue, the third vertex red, 
and so on. The nth vertex can be colored blue, because the two vertices adjacent to it, namely 
the(n — 1)st and the first vertices, are both colored red. 

When n is odd and n > 1, the chromatic number of C, is 3. To see this, pick an initial vertex. 
To use only two colors, it is necessary to alternate colors as the graph is traversed in a clockwise 
direction. However, the nth vertex reached is adjacent to two vertices of different colors, namely, 
the first and (n — 1)st. Hence, a third color must be used. 

We have shown that 7(C,,) = 2 if n is an even positive integer with n > 4 and 7(C,) = 3 if 
n is an odd positive integer with n > 3. 


The best algorithms known for finding the chromatic number of a graph have exponential 
worst-case time complexity (in the number of vertices of the graph). Even the problem of finding 
an approximation to the chromatic number of a graph is difficult. It has been shown that if 
there were an algorithm with polynomial worst-case time complexity that could approximate 
the chromatic number of a graph up to a factor of 2 (that is, construct a bound that was no 
more than double the chromatic number of the graph), then an algorithm with polynomial worst- 
case time complexity for finding the chromatic number of the graph would also exist. 


10.8.2 Applications of Graph Colorings 


Graph coloring has a variety of applications to problems involving scheduling and assignments. 
(Note that because no efficient algorithm is known for graph coloring, this does not lead to 
efficient algorithms for scheduling and assignments.) Examples of such applications will be 
given here. The first application deals with the scheduling of final exams. 


Scheduling Final Exams How can the final exams at a university be scheduled so that no 
student has two exams at the same time? 
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1 Red 
1 
Brown 7 2 Blue 
7 2 Time Period Courses 
I 1,6 
II 2 
Tl 3,5 
6 3 Red 6 3 Green IV 4,7 
5 4 5 Green 4 Brown 
FIGURE 8 The graph FIGURE 9 Using a coloring to schedule final exams. 


representing the scheduling 


of final exams. 


EXAMPLE 6 


EXAMPLE 7 


Solution: This scheduling problem can be solved using a graph model, with vertices representing 
courses and with an edge between two vertices if there is a common student in the courses they 
represent. Each time slot for a final exam is represented by a different color. A scheduling of the 
exams corresponds to a coloring of the associated graph. 

For instance, suppose there are seven finals to be scheduled. Suppose the courses are num- 
bered 1 through 7. Suppose that the following pairs of courses have common students: 1 and 2, 
1 and 3, 1 and 4, 1 and 7, 2 and 3, 2 and 4, 2 and 5, 2 and 7, 3 and 4, 3 and 6, 3 and 7, 4 and 5, 
4 and 6, 5 and 6, 5 and 7, and 6 and 7. In Figure 8 the graph associated with this set of classes 
is shown. A scheduling consists of a coloring of this graph. 

Because the chromatic number of this graph is 4 (the reader should verify this), four time 
slots are needed. A coloring of the graph using four colors and the associated schedule are shown 
in Figure 9. < 


Now consider an application to the assignment of television channels. 


Frequency Assignments Television channels 2 through 13 are assigned to stations in North 
America so that no two stations within 150 miles can operate on the same channel. How can 
the assignment of channels be modeled by graph coloring? 


Solution: Construct a graph by assigning a vertex to each station. Two vertices are connected 
by an edge if they are located within 150 miles of each other. An assignment of channels corre- 
sponds to a coloring of the graph, where each color represents a different channel. < 


An application of graph coloring to compilers is considered in Example 7. 


Index Registers In efficient compilers the execution of loops is speeded up when frequently 
used variables are stored temporarily in index registers in the central processing unit, instead 
of in regular memory. For a given loop, how many index registers are needed? This problem 
can be addressed using a graph coloring model. To set up the model, let each vertex of a graph 
represent a variable in the loop. There is an edge between two vertices if the variables they 
represent must be stored in index registers at the same time during the execution of the loop. 
Thus, the chromatic number of the graph gives the number of index registers needed, because 
different registers must be assigned to variables when the vertices representing these variables 
are adjacent in the graph. < 
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Exercises 


In Exercises 1—4 construct the dual graph for the map shown. 
Then find the number of colors needed to color the map so 
that no two adjacent regions have the same color. 


OP 


D 


In Exercises 5-11 find the chromatic number of the given 


graph. 


4 


we 
SN 


f 


bd, 
ei 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


a b 


m 


b c k l 


For the graphs in Exercises 5-11, decide whether it is 
possible to decrease the chromatic number by removing 
a single vertex and all edges incident with it. 


Which graphs have a chromatic number of 1? 


What is the least number of colors needed to color a 
map of the United States? Do not consider adjacent states 
that meet only at a corner. Suppose that Michigan is one 
region. Consider the vertices representing Alaska and 
Hawaii as isolated vertices. 


What is the chromatic number of W,,? 


Show that a simple graph that has a circuit with an odd 
number of vertices in it cannot be colored using two 
colors. 


Schedule the final exams for Math 115, Math 116, 
Math 185, Math 195, CS 101, CS 102, CS 273, and 
CS 473, using the fewest number of different time slots, 
if there are no students taking both Math 115 and CS 473, 
both Math 116 and CS 473, both Math 195 and CS 101, 
both Math 195 and CS 102, both Math 115 and Math 
116, both Math 115 and Math 185, and both Math 185 
and Math 195, but there are students in every other pair 
of courses. 


18. How many different channels are needed for six stations 
located at the distances shown in the table, if two sta- 
tions cannot use the same channel when they are within 
150 miles of each other? 


5 
reco [10 [20 [2 [1 


19. The mathematics department has six committees, each 
meeting once a month. How many different meeting 
times must be used to ensure that no member is scheduled 
to attend two meetings at the same time if the committees 
are C} = {Arlinghaus, Brand, Zaslavsky}, C) = {Brand, 
Lee, Rosen}, C} = {Arlinghaus, Rosen, Zaslavsky}, 
C, = {Lee, Rosen, Zaslavsky}, C; = {Arlinghaus, 
Brand}, and Cs = {Brand, Rosen, Zaslavsky}? 


20. A zoo wants to set up natural habitats in which to exhibit 
its animals. Unfortunately, some animals will eat some of 
the others when given the opportunity. How can a graph 
model and a coloring be used to determine the number 
of different habitats needed and the placement of the an- 
imals in these habitats? 


An edge coloring of a graph is an assignment of colors to 
edges so that edges incident with a common vertex are as- 
signed different colors. The edge chromatic number of a 
graph is the smallest number of colors that can be used in 
an edge coloring of the graph. The edge chromatic number of 
a graph G is denoted by y/(G). 


21. Find the edge chromatic number of each of the graphs in 
Exercises 5-11. 


22. Suppose that n devices are on a circuit board and that 
these devices are connected by colored wires. Express 
the number of colors needed for the wires, in terms of the 
edge chromatic number of the graph representing this cir- 
cuit board, under the requirement that the wires leaving 
a particular device must be different colors. Explain your 
answer. 


23. Find the edge chromatic numbers of 
a) C,,, where n > 3. 


n?’ 


b) W,,, where n > 3. 


n? 


24. Show that the edge chromatic number of a graph must be 
at least as large as the maximum degree of a vertex of the 
graph. 
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25. Show that if G is a graph with n vertices, then no more 
than n/2 edges can be colored the same in an edge color- 
ing of G. 


* 26. Find the edge chromatic number of K,, when n is a posi- 
tive integer. 


27. Seven variables occur in a loop of a computer program. 
The variables and the steps during which they must be 
stored are t: steps 1 through 6; u: step 2; v: steps 2 through 
4; w: steps 1, 3, and 5; x: steps 1 and 6; y: steps 3 through 
6; and z: steps 4 and 5. How many different index regis- 
ters are needed to store these variables during execution? 


28. What can be said about the chromatic number of a graph 
that has K,, as a subgraph? 


This algorithm can be used to color a simple graph: First, list 
the vertices v,, vy, V3, ..., V, in order of decreasing degree so 
that deg(v,) > deg(v,) > =- > deg(v,,). Assign color 1 to vı 
and to the next vertex in the list not adjacent to v, (if one ex- 
ists), and successively to each vertex in the list not adjacent 
to a vertex already assigned color 1. Then assign color 2 to 
the first vertex in the list not already colored. Successively as- 
sign color 2 to vertices in the list that have not already been 
colored and are not adjacent to vertices assigned color 2. If 
uncolored vertices remain, assign color 3 to the first vertex in 
the list not yet colored, and use color 3 to successively color 
those vertices not already colored and not adjacent to vertices 
assigned color 3. Continue this process until all vertices are 
colored. 


29. Construct a coloring of the graph shown using this algo- 
rithm. 


og 
> 
— 


* 30. Use pseudocode to describe this coloring algorithm. 


* 31. Show that the coloring produced by this algorithm may 
use more colors than are necessary to color a graph. 


A connected graph G is called chromatically k-critical if the 
chromatic number of G is k, but for every edge of G, the chro- 
matic number of the graph obtained by deleting this edge from 
Gisk-1. 


32. Show that C, is chromatically 3-critical whenever n is an 
odd positive integer, n > 3. 


33. Show that W, is chromatically 4-critical whenever n is an 
odd integer, n > 3. 


34. Show that W, is not chromatically 3-critical. 
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35. Show that if G is a chromatically k-critical graph, then 
the degree of every vertex of G is at least k — 1. 


A k-tuple coloring of a graph G is an assignment of a set of k 
different colors to each of the vertices of G such that no two 
adjacent vertices are assigned a common color. We denote by 
X{G) the smallest positive integer n such that G has a k-tuple 
coloring using n colors. For example, y (C4) = 4. To see this, 
note that using only four colors we can assign two colors to 
each vertex of C4, as illustrated, so that no two adjacent ver- 
tices are assigned the same color. Furthermore, no fewer than 
four colors suffice because the vertices v; and v, each must be 
assigned two colors, and a common color cannot be assigned 
to both v; and v,. (For more information about k-tuple color- 
ing, see [MiRo91].) 


{red, blue} v; v {green, yellow} 


{green, yellow} v4 v3 {red, blue} 


36. Find these values: 


a) %(K3) b) %(K4) ©) %(W4) 
d) 7(Cs) e) %2(K34) f) 73(Ks) 
*g) 43(Cs) h) 13(K45) 


*37. Let G and H be the graphs displayed in Figure 3. Find 
a) 42(G). b) x(H). 


©) 73(G). d) 73(H). 


38. What is y,(G) if G is a bipartite graph and k is a positive 
integer? 


39. Frequencies for mobile radio (or cellular) telephones are 
assigned by zones. Each zone is assigned a set of fre- 
quencies to be used by vehicles in that zone. The same 
frequency cannot be used in different zones when in- 
terference can occur between telephones in these zones. 
Explain how a k-tuple coloring can be used to assign k 
frequencies to each mobile radio zone in a region. 


* 40. Show that every planar graph G can be colored using six 
or fewer colors. [Hint: Use mathematical induction on 
the number of vertices of the graph. Apply Corollary 2 
of Section 10.7 to find a vertex v with deg(v) < 5. Con- 
sider the subgraph of G obtained by deleting v and all 
edges incident with it.] 


** 41. Show that every planar graph G can be colored using five 


or fewer colors. [Hint: Use the hint provided for Exer- 
cise 40.] 


The art gallery problem asks how many guards are needed to 
see all parts of an art gallery, where the gallery is the interior 
and boundary of a polygon with n sides. To state this problem 
more precisely, we need some terminology. A point x inside 
or on the boundary of a simple polygon P covers or sees a 
point y inside or on P if all points on the line segment xy are 
in the interior or on the boundary of P. We say that a set of 
points is a guarding set of a simple polygon P if for every 
point y inside P or on the boundary of P there is a point x in 
this guarding set that sees y. Denote by G(P) the minimum 
number of points needed to guard the simple polygon P. The 
art gallery problem asks for the function g(n), which is the 
maximum value of G(P) over all simple polygons with n ver- 
tices. That is, g(n) is the minimum positive integer for which 
it is guaranteed that a simple polygon with n vertices can be 
guarded with g(n) or fewer guards. 


42. Show that g(3) = 1 and g(4) = 1 by showing that all 
triangles and quadrilaterals can be guarded using one 
point. 


* 43. Show that g(5) = 1. That is, show that all pentagons 
can be guarded using one point. [Hint: Show that there 
are either 0, 1, or 2 vertices with an interior angle 
greater than 180 degrees and that in each case, one guard 
suffices. ] 


*44. Show that g(6) = 2 by first using Exercises 42 and 43 as 
well as Lemma 1 in Section 5.2 to show that 9(6) < 2 
and then find a simple hexagon for which two guards are 
needed. 


*45. Show that g(n) > [n/3]. [Hint: Consider the polygon 
with 3k vertices that resembles a comb with k prongs, 
such as the polygon with 15 sides shown here.] 


AAAAA 


* 46. Solve the art gallery problem by proving the art gallery 
theorem, which states that at most |n/3] guards are 
needed to guard the interior and boundary of a simple 
polygon with n vertices. [Hint: Use Theorem 1 in Sec- 
tion 5.2 to triangulate the simple polygon into n — 2 tri- 
angles. Then show that it is possible to color the vertices 
of the triangulated polygon using three colors so that no 
two adjacent vertices have the same color. Use induction 
and Exercise 23 in Section 5.2. Finally, put guards at all 
vertices that are colored red, where red is the color used 
least in the coloring of the vertices. Show that placing 
guards at these points is all that is needed.] 


Key Terms and Results 


TERMS 


undirected edge: an edge associated to a set {u, v}, where u 
and v are vertices 

directed edge: an edge associated to an ordered pair (u, v), 
where u and v are vertices 


multiple edges: distinct edges connecting the same vertices 


multiple directed edges: distinct directed edges associated 
with the same ordered pair (u,v), where u and v are vertices 


loop: an edge connecting a vertex with itself 

undirected graph: a set of vertices and a set of undirected 
edges each of which is associated with a set of one or two 
of these vertices 

simple graph: an undirected graph with no multiple edges or 
loops 

multigraph: an undirected graph that may contain multiple 
edges but no loops 

pseudograph: an undirected graph that may contain multiple 
edges and loops 

directed graph: a set of vertices together with a set of directed 
edges each of which is associated with an ordered pair of 
vertices 


directed multigraph: a graph with directed edges that may 
contain multiple directed edges 


simple directed graph: a directed graph without loops or mul- 
tiple directed edges 

adjacent: two vertices are adjacent if there is an edge between 
them 


incident: an edge is incident with a vertex if the vertex is an 
endpoint of that edge 

deg v (degree of the vertex v in an undirected graph): the 
number of edges incident with v with loops counted twice 

deg’ (v) (the in-degree of the vertex v in a graph with di- 
rected edges): the number of edges with v as their terminal 
vertex 


deg* (v) (the out-degree of the vertex v in a graph with di- 
rected edges): the number of edges with v as their initial 
vertex 

underlying undirected graph of a graph with directed 
edges: the undirected graph obtained by ignoring the di- 
rections of the edges 

K,, (complete graph on n vertices): the undirected graph with 
n vertices where each pair of vertices is connected by an 
edge 

bipartite graph: a graph with vertex set that can be partitioned 
into subsets V, and V, so that each edge connects a vertex 
in V, and a vertex in V,. The pair (V,, V>) is called a bipar- 
tition of V. 

Knn (complete bipartite graph): the graph with vertex set 
partitioned into a subset of m elements and a subset of n el- 
ements with two vertices connected by an edge if and only 
if one is in the first subset and the other is in the second 
subset 
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C,, (cycle of size n), n>3: the graph with n vertices 
Vis Va, -.-, Vp and edges {v,, Vo}, {vz v3 h -> (Veep Ynh 
(Vp vı } 

W,, (wheel of size n), n > 3: the graph obtained from C, by 
adding a vertex and edges from this vertex to the original 
vertices in C, 

Q,, (n-cube), n > 1: the graph that has the 2” bit strings of 
length n as its vertices and edges connecting every pair of 
bit strings that differ by exactly one bit 

matching in a graph G: a set of edges such that no two edges 
have a common endpoint 

complete matching M from V; to Vj: a matching such that 
every vertex in V, is an endpoint of an edge in M 

maximum matching: a matching containing the most edges 
among all matchings in a graph 

isolated vertex: a vertex of degree zero 

pendant vertex: a vertex of degree one 

regular graph: a graph where all vertices have the same de- 
gree 

subgraph of a graph G = (V, E): a graph (W, F), where W is 
a subset of V and F is a subset of E 

G, UG, (union of G} and G,): the graph (V; U V), E; U E3), 
where G, = (V,, E,) and G, = (V9, E3) 

adjacency matrix: a matrix representing a graph using the ad- 
jacency of vertices 

incidence matrix: a matrix representing a graph using the in- 
cidence of edges and vertices 

isomorphic simple graphs: the simple graphs G, = (V,, E;) 
and G, = (V, E) are isomorphic if there exists a one- 
to-one correspondence f from V, to V, such that 
{f(v,), f(v2)} © E if and only if {v,, v2} € £ forall v, and 
v, in V} 

invariant for graph isomorphism: a property that isomor- 
phic graphs either both have or both do not have 

path from u to v in an undirected graph: a sequence of 
edges e}, €z, ..., €„, Where e; is associated to {x;, x;,,} for 
i= 0, 1, ...,n, where x) =uandx,,, = V 

path from u to v in a graph with directed edges: a sequence 
of edges e}, ez, ..., €„», Where e; is associated to (x;, x;,,) for 
i=0,1,...,n, where x) =uandx,,, = V 

simple path: a path that does not contain an edge more than 
once 

circuit: a path of length n > 1 that begins and ends at the same 
vertex 

connected graph: an undirected graph with the property that 
there is a path between every pair of vertices 

cut vertex of G: a vertex v such that G — v is disconnected 

cut edge of G: an edge e such that G — e is disconnected 

nonseparable graph: a graph without a cut vertex 

vertex cut of G: a subset V’ of the set of vertices of G such 
that G — V’ is disconnected 

k(G) (the vertex connectivity of G): the size of a smallest 
vertex cut of G 
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k-connected graph: a graph that has a vertex connectivity no 
smaller than k 

edge cut of G: a set of edges E’ of G such that G — E’ is dis- 
connected 

A(G) (the edge connectivity of G): the size of a smallest edge 
cut of G 

connected component of a graph G: a maximal connected 
subgraph of G 

strongly connected directed graph: a directed graph with the 
property that there is a directed path from every vertex to 
every vertex 

strongly connected component of a directed graph G: a 
maximal strongly connected subgraph of G 

Euler path: a path that contains every edge of a graph exactly 
once 

Euler circuit: a circuit that contains every edge of a graph 
exactly once 

Hamilton path: a path in a graph that passes through each 
vertex exactly once 

Hamilton circuit: a circuit in a graph that passes through each 
vertex exactly once 

weighted graph: a graph with numbers assigned to its edges 

shortest-path problem: the problem of determining the path 
in a weighted graph such that the sum of the weights of 
the edges in this path is a minimum over all paths between 
specified vertices 

traveling salesperson problem: the problem that asks for the 
circuit of shortest total length that visits every vertex of a 
weghted graph exactly once 

planar graph: a graph that can be drawn in the plane with no 
crossings 

regions of a representation of a planar graph: the regions 
the plane is divided into by the planar representation of the 
graph 


Review Questions 


1. a) Define a simple graph, a multigraph, a pseudograph, 
a directed graph, and a directed multigraph. 

b) Use an example to show how each of the types of 
graph in part (a) can be used in modeling. For exam- 
ple, explain how to model different aspects of a com- 
puter network or airline routes. 

2. Give at least four examples of how graphs are used in 
modeling. 

3. What is the relationship between the sum of the degrees 
of the vertices in an undirected graph and the number of 
edges in this graph? Explain why this relationship holds. 

4. Why must there be an even number of vertices of odd de- 
gree in an undirected graph? 

5. What is the relationship between the sum of the in- 
degrees and the sum of the out-degrees of the vertices 
in a directed graph? Explain why this relationship holds. 


elementary subdivision: the removal of an edge {u, v} of an 
undirected graph and the addition of a new vertex w to- 
gether with edges {u, w} and {w, v} 

homeomorphic: two undirected graphs are homeomorphic if 
they can be obtained from the same graph by a sequence of 
elementary subdivisions 

graph coloring: an assignment of colors to the vertices of a 
graph so that no two adjacent vertices have the same color 


chromatic number: the minimum number of colors needed 
in a coloring of a graph 


RESULTS 


The handshaking theorem: If G = (V, E) be an undirected 
graph with m edges, then 2m = J, „ey deg(v). 

Hall’s marriage theorem: The bipartite graph G = (V, E) 
with bipartition (V,, V,) has a complete matching from V, 
to V, if and only if |N(A)| > |A] for all subsets A of V}. 


There is an Euler circuit in a connected multigraph if and only 
if every vertex has even degree. 


There is an Euler path in a connected multigraph if and only if 
at most two vertices have odd degree. 


Dijkstra’s algorithm: a procedure for finding a shortest path 
between two vertices in a weighted graph (see Section 10.6) 

Euler’s formula: r = e — v + 2 where r, e, and v are the num- 
ber of regions of a planar representation, the number of 
edges, and the number of vertices, respectively, of a con- 
nected planar graph 

Kuratowski’s theorem: A graph is nonplanar if and only if 
it contains a subgraph homeomorphic to K; 3 or K;. (Proof 
beyond the scope of this book.) 


The four color theorem: Every planar graph can be colored 
using no more than four colors. (Proof far beyond the scope 
of this book!) 


6. Describe the following families of graphs. 
a) K 


n?’ 


b) K 


m,n? 


the complete graph on n vertices 

the complete bipartite graph on m and n vertices 
c) C, the cycle with n vertices 

d) W,, the wheel of size n 


n? 


e) Q, the n-cube 


7. How many vertices and how many edges are there in each 
of the graphs in the families in Question 6? 

8. a) What is a bipartite graph? 
b) Which of the graphs K,,, C 


c) How can you determine whether an undirected graph 
is bipartite? 


and W, are bipartite? 


n? 


9. a) Describe three different methods that can be used to 
represent a graph. 


10. 


11. 


12. 


13. 


b) Draw a simple graph with at least five vertices and 
eight edges. Illustrate how it can be represented using 
the methods you described in part (a). 


a) What does it mean for two simple graphs to be iso- 
morphic? 

b) What is meant by an invariant with respect to isomor- 
phism for simple graphs? Give at least five examples 
of such invariants. 

c) Give an example of two graphs that have the same 
numbers of vertices, edges, and degrees of vertices, 
but that are not isomorphic. 

d) Is a set of invariants known that can be used to effi- 
ciently determine whether two simple graphs are iso- 
morphic? 

a) What does it mean for a graph to be connected? 

b) What are the connected components of a graph? 


a) Explain how an adjacency matrix can be used to rep- 
resent a graph. 

b) How can adjacency matrices be used to determine 
whether a function from the vertex set of a graph G 
to the vertex set of a graph H is an isomorphism? 

c) How can the adjacency matrix of a graph be used to 
determine the number of paths of length r, where r is 
a positive integer, between two vertices of a graph? 


a) Define an Euler circuit and an Euler path in an undi- 
rected graph. 

b) Describe the famous Königsberg bridge problem and 
explain how to rephrase it in terms of an Euler circuit. 

c) How can it be determined whether an undirected 
graph has an Euler path? 


Supplementary Exercises 


1. 


2. 


How many edges does a 50-regular graph with 100 ver- 
tices have? 


How many nonisomorphic subgraphs does K, have? 


In Exercises 3—5 determine whether two given graphs are iso- 
morphic. 


3. 


uy Uy U3 U4 


us Ug uz ug 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 
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d) How can it be determined whether an undirected 
graph has an Euler circuit? 

a) Define a Hamilton circuit in a simple graph. 

b) Give some properties of a simple graph that imply that 
it does not have a Hamilton circuit. 

Give examples of at least two problems that can be solved 

by finding the shortest path in a weighted graph. 

a) Describe Dijkstra’s algorithm for finding the shortest 
path in a weighted graph between two vertices. 

b) Draw a weighted graph with at least 10 vertices and 
20 edges. Use Dijkstra’s algorithm to find the shortest 
path between two vertices of your choice in the graph. 

a) What does it mean for a graph to be planar? 

b) Give an example of a nonplanar graph. 

a) What is Euler’s formula for connected planar graphs? 

b) How can Euler’s formula for planar graphs be used to 
show that a simple graph is nonplanar? 

State Kuratowski’s theorem on the planarity of graphs 

and explain how it characterizes which graphs are planar. 

a) Define the chromatic number of a graph. 

b) What is the chromatic number of the graph K, when 
n is a positive integer? 

c) What is the chromatic number of the graph C, when 
n is an integer greater than 2? 

d) What is the chromatic number of the graph K, „ when 
m and n are positive integers? 

State the four color theorem. Are there graphs that cannot 

be colored with four colors? 

Explain how graph coloring can be used in modeling. Use 

at least two different examples. 


ug uy 
u7 U3 


Ug ug 
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u7 u3 
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The complete m-partite graph K, „n, has vertices parti- 
tioned into m subsets of 1, n, ..., 71, elements each, and ver- 
tices are adjacent if and only if they are in different subsets in 


the partition. 
6. Draw these graphs. 


a) Ki23 b) Ky 59 


*7. How many vertices and how many edges does the com- 
plete m-partite graph K, have? 


c) Ky 593 


[olay ++) 

8. Prove or disprove that there are always two vertices of 
the same degree in a finite multigraph having at least two 
vertices. 


9. Let G = (V, E) be an undirected graph and let A C V and 
B C V. Show that 
a) N(A U B) = N(A) U N(B). 
b) N(A NB) C N(A) NA N(B), and give an example where 
N(A N B) # N(A) A N(B). 
10. Let G = (V, E) be an undirected graph. Show that 
a) |N(v)| < deg(v) for all v € V. 
b) |N()| = degv for all v € V if and only if G is a simple 
graph. 
Suppose that S$}, Sz, ..., S is a collection of subsets of a 
set § where n is a positive integer. A system of distinct 
representatives (SDR) for this family is an ordered n-tuple 
(a), do, ..., a,,) With the property thata; € S; fori = 1,2, ..., n 
and a; # a; for all i # j. 
11. Find a SDR for the sets S; = {a,c,m,e}, S= 
{mace}, S;={ap,ex}, Sy={xena}, Ss= 
{n, a,m, e}, and S = {e, x, a, m}. 
12. Use Hall’s marriage theorem to show that a collection 
of finite subsets S4, S2, ..., S, of a set S has a SDR 
(a, a, ...,a,) if and only if | Ue S;l = H| for all sub- 
sets J of {1, 2, ..., n}. 


13. a) Use Exercise 12 to show that the collection of 
sets $; = {a,b,c}, S, = {b, c, d}, 8, = {a, b, d}, Sy = 
{b, c, d} has a SDR without finding one explicitly. 


b) Find a SDR for the family of four sets in part (a). 


14. Use Exercise 12 to show that collection of sets S; = 
{a,b,c}, S, = {a,c}, S3 = {c,d e}, S4 = {b,c}, S; = 
{d e, f}, S6 = {a, c, e}, and S = {a,b} does not have a 
SDR. 

The clustering coefficient C(G) of a simple graph G is the 

probability that if u and v are neighbors and v and w are neigh- 

bors, then u and w are neighbors, where u, v, and w are distinct 

vertices of G. 


15. We say that three vertices u, v, and w of a simple graph 
G form a triangle if there are edges connecting all three 
pairs of these vertices. Find a formula for C(G) in terms 
of the number of triangles in G and the number of paths 
of length two in the graph. [Hint: Count each triangle 
in the graph once for each order of three vertices that 
form it.] 


16. Find the clustering coefficient of each of the graphs in 
Exercise 20 of Section 10.2. 


17. Explain what the clustering coefficient measures in each 
of these graphs. 


a) the Hollywood graph 
b) the graph of Facebook friends 


c) the academic collaboration graph for researchers in 
graph theory 


d) the protein interaction graph for a human cell 


e) the graph representing the routers and communica- 
tions links that make up the worldwide Internet 
18. For each of the graphs in Exercise 17, explain whether 
you would expect its clustering coefficient to be closer to 
0.01 or to 0.10 and why you expect this. 
A clique in a simple undirected graph is a complete subgraph 
that is not contained in any larger complete subgraph. In Ex- 
ercises 19-21 find all cliques in the graph shown. 


21. 


h g f 


A dominating set of vertices in a simple graph is a set of ver- 
tices such that every other vertex is adjacent to at least one 
vertex of this set. A dominating set with the least number of 
vertices is called a minimum dominating set. In Exercises 
22-24 find a minimum dominating set for the given graph. 


22. 23. 
a b a e 
XxX | | : | 
c d b f 


24. a b c d 


i j k l 

A simple graph can be used to determine the minimum num- 
ber of queens on a chessboard that control the entire chess- 
board. An n Xn chessboard has n? squares in an n X n con- 
figuration. A queen in a given position controls all squares 
in the same row, the same column, and on the two diagonals 
containing this square, as illustrated. The appropriate simple 
graph has n? vertices, one for each square, and two vertices 
are adjacent if a queen in the square represented by one of the 
vertices controls the square represented by the other vertex. 


cee 


et 


The squares 
controlled 
by a queen. 
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25. Construct the simple graph representing the n X n chess- 
board with edges representing the control of squares by 
queens for 
a) n=3. b) n=4. 

26. Explain how the concept of a minimum dominating set 
applies to the problem of determining the minimum num- 
ber of queens controlling an n x n chessboard. 


** 27, Find the minimum number of queens controlling ann X n 


chessboard for 


a) n=3. b) n=4. c) n=5. 

28. Suppose that G, and H, are isomorphic and that G, and 
H, are isomorphic. Prove or disprove that G) U G, and 
H, U H, are isomorphic. 

29. Show that each of these properties is an invariant that iso- 
morphic simple graphs either both have or both do not 
have. 

a) connectedness 
b) the existence of a Hamilton circuit 
c) the existence of an Euler circuit 
d) having crossing number C 
e) having n isolated vertices 
f) being bipartite 
30. How can the adjacency matrix of G be found from the 


adjacency matrix of G, where G is a simple graph? 


31. How many nonisomorphic connected bipartite simple 
graphs are there with four vertices? 


*32. How many nonisomorphic simple connected graphs with 


five vertices are there 

a) with no vertex of degree more than two? 
b) with chromatic number equal to four? 

c) that are nonplanar? 


A directed graph is self-converse if it is isomorphic to its con- 
verse. 


33. Determine whether the following graphs are self- 
converse. 


a) a b 


b) 
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34. Show that if the directed graph G is self-converse and H 
is a directed graph isomorphic to G, then H is also self- 
converse. 


An orientation of an undirected simple graph is an assign- 
ment of directions to its edges such that the resulting di- 
rected graph is strongly connected. When an orientation of 
an undirected graph exists, this graph is called orientable. In 
Exercises 35-37 determine whether the given simple graph is 
orientable. 


35. a b c 
ə 
e d 
36. a f 
b c d e 
37. a b e 
c 
d f 
h g 


38. Because traffic is growing heavy in the central part of 
a city, traffic engineers are planning to change all the 
streets, which are currently two-way, into one-way streets. 
Explain how to model this problem. 


* 39. Show that a graph is not orientable if it has a cut edge. 
A tournament is a simple directed graph such that if u and 
v are distinct vertices in the graph, exactly one of (u, v) and 
©, u) is an edge of the graph. 


40. How many different tournaments are there with n ver- 
tices? 


41. What is the sum of the in-degree and out-degree of a ver- 
tex in a tournament? 


* 42. Show that every tournament has a Hamilton path. 
43. Given two chickens in a flock, one of them is dominant. 


This defines the pecking order of the flock. How can a 
tournament be used to model pecking order? 


44. Suppose that a connected graph G has n vertices and ver- 
tex connectivity x(G) = k. Show that G must have at least 
[kn/2] edges. 


A connected graph G = (V, E) with n vertices and m edges 

is said to have optimal connectivity if «(G) = A(G) = 

Min, cy deg v = 2m/n. 

45. Show that a connected graph with optimal connectivity 
must be regular. 


46. Show these graphs have optimal connectivity. 
a) C, forn > 3 
b) K,, forn > 3 
c) K,, forr > 2 


*47, Find the two nonisomorphic simple graphs with six ver- 
tices and nine edges that have optimal connectivity. 


48. Suppose that G is a connected multigraph with 2k ver- 
tices of odd degree. Show that there exist k subgraphs that 
have G as their union, where each of these subgraphs has 
an Euler path and where no two of these subgraphs have 
an edge in common. [Hint: Add k edges to the graph con- 
necting pairs of vertices of odd degree and use an Euler 
circuit in this larger graph. ] 


In Exercises 49 and 50 we consider a puzzle posed by Petković 
in [Pe09] (based on a problem in [AvCh80]). Suppose that 
King Arthur has gathered his 2” knights of the Round Table 
for an important council. Every two knights are either friends 
or enemies, and each knight has no more than n — | enemies 
among the other 2n — 1 knights. The puzzle asks whether 
King Arthur can seat his knights around the Round Table so 
that each knight has two friends for his neighbors. 


49. a) Show that the puzzle can be reduced to determining 
whether there is a Hamilton circuit in the graph in 
which each knight is represented by a vertex and two 
knights are connected in the graph if they are friends. 

b) Answer the question posed in the puzzle. [Hint: Use 
Dirac’s theorem. ] 


50. Suppose that are eight knights Alynore, Bedivere, De- 
gore, Gareth, Kay, Lancelot, Perceval, and Tristan. 
Their lists of enemies are A (D, G, P), B (K, P, T), D 
(A, G, L), G (A, D, T), K (B, L, P), L (D, K, T), P (A, B, 
K), T (B, G, L), where we have represented each knight by 
the first letter of his name and shown the list of enemies of 
that knight following this first letter. Draw the graph rep- 
resenting these eight knights and their friends and find a 
seating arrangement where each knight sits next to two 
friends. 


*51. Let G be a simple graph with n vertices. The bandwidth 
of G, denoted by B(G), is the minimum, over all permuta- 
tions 4}, a», ..., a, Of the vertices of G, of max{ |i — j| la; 
and aj are adjacent}. That is, the bandwidth is the mini- 
mum over all listings of the vertices of the maximum dif- 
ference in the indices assigned to adjacent vertices. Find 
the bandwidths of these graphs. 

a) K; b) K,3 c) Ky, 
d) K3 e) Q; f) C; 


* 52. 


#53: 


* 54. 


55. 


56. 


57. 


58. 


* 59, 


The distance between two distinct vertices v, and v, of a 
connected simple graph is the length (number of edges) of 
the shortest path between v, and v,. The radius of a graph 
is the minimum over all vertices v of the maximum dis- 
tance from v to another vertex. The diameter of a graph 
is the maximum distance between two distinct vertices. 
Find the radius and diameter of 


a) Kg. b) Kys. c) Q}. 


a) Show that if the diameter of the simple graph G is at 
least four, then the diameter of its complement Gis no 
more than two. 

b) Show that if the diameter of the simple graph G is at 
least three, then the diameter of its complement Gis 
no more than three. 


d) Cy. 


Suppose that a multigraph has 2m vertices of odd de- 
gree. Show that any circuit that contains every edge of 
the graph must contain at least m edges more than once. 

Find the second shortest path between the vertices a and 
z in Figure 3 of Section 10.6. 

Devise an algorithm for finding the second shortest path 
between two vertices in a simple connected weighted 
graph. 

Find the shortest path between the vertices a and z that 
passes through the vertex f in the weighted graph in Ex- 
ercise 3 in Section 10.6. 

Devise an algorithm for finding the shortest path between 
two vertices in a simple connected weighted graph that 
passes through a specified third vertex. 

Show that if G is a simple graph with at least 11 vertices, 
then either G or G, the complement of G, is nonplanar. 


> A set of vertices in a graph is called independent if no two 
vertices in the set are adjacent. The independence number of 
a graph is the maximum number of vertices in an independent 
set of vertices for the graph. 


* 60. 


61. 


What is the independence number of 
a) K,? b) C,? c) Q,? d) Kin? 
Show that the number of vertices in a simple graph is less 
than or equal to the product of the independence number 
and the chromatic number of the graph. 


Computer Projects 


Write programs with these input and output. 


. Given the vertex pairs associated to the edges of an undi- 


rected graph, find the degree of each vertex. 


. Given the ordered pairs of vertices associated to the edges 


of a directed graph, determine the in-degree and out- 
degree of each vertex. 


. Given the list of edges of a simple graph, determine 


whether the graph is bipartite. 


. Given the vertex pairs associated to the edges of a graph, 


construct an adjacency matrix for the graph. (Produce 


62. 


63. 
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Show that the chromatic number of a graph is less than 
or equal to n — i + 1, where n is the number of vertices in 
the graph and i is the independence number of this graph. 


Suppose that to generate a random simple graph with n 

vertices we first choose a real number p with O < p < 1. 

For each of the C(, 2) pairs of distinct vertices we gener- 

ate a random number x between 0 and 1. If 0 < x < p, we 

connect these two vertices with an edge; otherwise these 
vertices are not connected. 

a) What is the probability that a graph with m edges 
where 0 < m < C(u, 2) is generated? 

b) What is the expected number of edges in a randomly 
generated graph with n vertices if each edge is in- 
cluded with probability p? 

c) Show that if p = 1/2 then every simple graph with n 
vertices is equally likely to be generated. 


A property retained whenever additional edges are added to 
a simple graph (without adding vertices) is called monotone 
increasing, and a property that is retained whenever edges are 
removed from a simple graph (without removing vertices) is 
called monotone decreasing. 


64. 


65. 


** 66, 


For each of these properties, determine whether it is 
monotone increasing and determine whether it is mono- 
tone decreasing. 

a) The graph G is connected. 

b) The graph G is not connected. 

c) The graph G has an Euler circuit. 

d) The graph G has a Hamilton circuit. 

e) The graph G is planar. 

f) The graph G has chromatic number four. 

g) The graph G has radius three. 

h) The graph G has diameter three. 

Show that the graph property P is monotone increasing if 
and only if the graph property Q is monotone decreasing 
where Q is the property of not having property P. 
Suppose that P is a monotone increasing property of sim- 
ple graphs. Show that the probability a random graph 
with n vertices has property P is a monotonic nondecreas- 
ing function of p, the probability an edge is chosen to be 
in the graph. 


a version that works when loops, multiple edges, or 
directed edges are present.) 


. Given an adjacency matrix of a graph, list the edges 


of this graph and give the number of times each edge 
appears. 


. Given the vertex pairs associated to the edges of an undi- 


rected graph and the number of times each edge appears, 
construct an incidence matrix for the graph. 


778 


10. 


11. 
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. Given an incidence matrix of an undirected graph, list its 


edges and give the number of times each edge appears. 


. Given a positive integer n, generate a simple graph with n 


vertices by producing an adjacency matrix for the graph 
so that all simple graphs with n vertices are equally likely 
to be generated. 


. Given a positive integer n, generate a simple directed 


graph with n vertices by producing an adjacency matrix 
for the graph so that all simple directed graphs with n ver- 
tices are equally likely to be generated. 


Given the lists of edges of two simple graphs with no 
more than six vertices, determine whether the graphs are 
isomorphic. 


Given an adjacency matrix of a graph and a positive in- 
teger n, find the number of paths of length n between two 
vertices. (Produce a version that works for directed and 
undirected graphs.) 


Given the list of edges of a simple graph, determine 
whether it is connected and find the number of connected 
components if it is not connected. 


Given the vertex pairs associated to the edges of a multi- 
graph, determine whether it has an Euler circuit and, if 


Computations and Explorations 


*14, 


#15, 


** 16, 


17. 


18. 


19. 


20. 


not, whether it has an Euler path. Construct an Euler path 
or circuit if it exists. 

Given the ordered pairs of vertices associated to the edges 
of a directed multigraph, construct an Euler path or Euler 
circuit, if such a path or circuit exists. 

Given the list of edges of a simple graph, produce a 
Hamilton circuit, or determine that the graph does not 
have such a circuit. 

Given the list of edges of a simple graph, produce a 
Hamilton path, or determine that the graph does not have 
such a path. 

Given the list of edges and weights of these edges of a 
weighted connected simple graph and two vertices in this 
graph, find the length of a shortest path between them 
using Dijkstra’s algorithm. Also, find a shortest path. 
Given the list of edges of an undirected graph, find a col- 
oring of this graph using the algorithm given in the exer- 
cise set of Section 10.8. 

Given a list of students and the courses that they are 
enrolled in, construct a schedule of final exams. 

Given the distances between pairs of television stations 
and the minimum allowable distance between stations, 
assign frequencies to these stations. 


Use a computational program or programs you have written to do these exercises. 


. Display all simple graphs with four vertices. 


. Display a full set of nonisomorphic simple graphs with 


six vertices. 


. Display a full set of nonisomorphic directed graphs with 


four vertices. 


. Generate at random 10 different simple graphs each with 


20 vertices so that each such graph is equally likely to be 
generated. 


. Construct a Gray code where the code words are bit 


strings of length six. 


. Construct knight’s tours on chessboards of various sizes. 


. Determine whether each of the graphs you generated in 


Exercise 4 of this set is planar. If you can, determine the 
thickness of each of the graphs that are not planar. 


. Determine whether each of the graphs you generated in 


Exercise 4 of this set is connected. If a graph is not con- 
nected, determine the number of connected components 
of the graph. 


9. 


10. 


11. 


** 12, 


* 13. 


14, 


Generate at random simple graphs with 10 vertices. Stop 
when you have constructed one with an Euler circuit. Dis- 
play an Euler circuit in this graph. 


Generate at random simple graphs with 10 vertices. Stop 
when you have constructed one with a Hamilton circuit. 
Display a Hamilton circuit in this graph. 


Find the chromatic number of each of the graphs you gen- 
erated in Exercise 4 of this set. 


Find the shortest path a traveling salesperson can take to 
visit each of the capitals of the 50 states in the United 
States, traveling by air between cities in a straight line. 


Estimate the probability that a randomly generated sim- 
ple graph with n vertices is connected for each posi- 
tive integer n not exceeding ten by generating a set of 
random simple graphs and determining whether each is 
connected. 


Work on the problem of determining whether the cross- 
ing number of K,, is 77, 79, or 81. It is known that it 
equals one of these three values. 


Writing Projects 


Respond to these with essays using outside sources. 


10. 


11. 


. Describe the origins and development of graph theory 


prior to the year 1900. 


. Discuss the applications of graph theory to the study of 


ecosystems 7 


. Discuss the applications of graph theory to sociology and 


psychology. 


. Discuss what can be learned by investigating the proper- 


ties of the web graph. 


. Explain what community structure is in a graph repre- 


senting a network, such as a social network, a computer 
network, an information network, or a biological network. 
Define what a community in such a graph is, and explain 
what communities represent in graphs representing the 
types of networks listed. 


. Describe some of the algorithms used to detect commu- 


nities in graphs representing networks of the types listed 
in Question 5. 


. Describe algorithms for drawing a graph on paper or on 


a display given the vertices and edges of the graph. What 
considerations arise in drawing a graph so that it has the 
best appearance for understanding its properties? 


. Explain how graph theory can help uncover networks 


of criminals or terrorists by studying relevant social and 
communication networks. 


. What are some of the capabilities that a software tool 


for inputting, displaying, and manipulating graphs should 
have? Which of these capabilities do available tools have? 


Describe some of the algorithms available for determin- 
ing whether two graphs are isomorphic and the computa- 
tional complexity of these algorithms. What is the most 
efficient such algorithm currently known? 


What is the subgraph isomorphism problem and what 
are some of its important applications, including those to 
chemistry, bioinformatics, electronic circuit design, and 
computer vision? 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 
25. 
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Explain what the area of graph mining, an important area 
of data mining, is and describe some of the basic tech- 
niques used in graph mining. 

Describe how Euler paths can be used to help determine 
DNA sequences. 

Define de Bruijn sequences and discuss how they arise 
in applications. Explain how de Bruijn sequences can be 
constructed using Euler circuits. 

Describe the Chinese postman problem and explain how 
to solve this problem. 

Describe some of the different conditions that imply that 
a graph has a Hamilton circuit. 

Describe some of the strategies and algorithms used to 
solve the traveling salesperson problem. 

Describe several different algorithms for determining 
whether a graph is planar. What is the computational 
complexity of each of these algorithms? 

In modeling, very large scale integration (VLSI) graphs 
are sometimes embedded in a book, with the vertices on 
the spine and the edges on pages. Define the book num- 
ber of a graph and find the book number of various graphs 
including K, for n = 3, 4, 5, and 6. 

Discuss the history of the four color theorem. 

Describe the role computers played in the proof of the 
four color theorem. How can we be sure that a proof that 
relies on a computer is correct? 

Describe and compare several different algorithms for 
coloring a graph, in terms of whether they produce a 
coloring with the least number of colors possible and in 
terms of their complexity. 

Explain how graph multicolorings can be used in a vari- 
ety of different models. 

Describe some of the applications of edge colorings. 
Explain how the theory of random graphs can be used in 
nonconstructive existence proofs of graphs with certain 
properties. 


11.1 Introduction 
to Trees 


11.2 Applications 
of Trees 


11.3 Tree Traversal 


11.4 Spanning 
Trees 


11.5 Minimum 
Spanning 
Trees 


Trees 


A connected graph that contains no simple circuits is called a tree. Trees were used as long 
ago as 1857, when the English mathematician Arthur Cayley used them to count certain 
types of chemical compounds. Since that time, trees have been employed to solve problems in 
a wide variety of disciplines, as the examples in this chapter will show. 

Trees are particularly useful in computer science, where they are employed in a wide range 
of algorithms. For instance, trees are used to construct efficient algorithms for locating items in 
a list. They can be used in algorithms, such as Huffman coding, that construct efficient codes 
saving costs in data transmission and storage. Trees can be used to study games such as checkers 
and chess and can help determine winning strategies for playing these games. Trees can be used 
to model procedures carried out using a sequence of decisions. Constructing these models can 
help determine the computational complexity of algorithms based on a sequence of decisions, 
such as sorting algorithms. 

Procedures for building trees containing every vertex of a graph, including depth-first search 
and breadth-first search, can be used to systematically explore the vertices of a graph. Exploring 
the vertices of a graph via depth-first search, also known as backtracking, allows for the system- 
atic search for solutions to a wide variety of problems, such as determining how eight queens 
can be placed on a chessboard so that no queen can attack another. 

We can assign weights to the edges of a tree to model many problems. For example, using 
weighted trees we can develop algorithms to construct networks containing the least expensive 
set of telephone lines linking different network nodes. 


Introduction to Trees 


Links » 


In Chapter 10 we showed how graphs can be used to model and solve many problems. In this 
chapter we will focus on a particular type of graph called a tree, so named because such graphs 
resemble trees. For example, family trees are graphs that represent genealogical charts. Family 
trees use vertices to represent the members of a family and edges to represent parent-child rela- 
tionships. The family tree of the male members of the Bernoulli family of Swiss mathematicians 
is shown in Figure 1. The undirected graph representing a family tree (restricted to people of 
just one gender and with no inbreeding) is an example of a tree. 


a ee 


Nikolaus 
(1623-1708) 


Jacob I Nikolaus Johann I 
(1654-1705) (1662-1716) (1667-1748) 


Nikolaus I Nikolaus II Daniel Johann II 
(1687-1759) (1695-1726) (1700-1782) (1710-1790) 


/ \ 


Johann M Jacob H 
(1746-1807) (1759-1789) 


FIGURE 1 The Bernoulli family of mathematicians. o 
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Definition 1 


EXAMPLE 1 


THEOREM 1 


a b a b a b a b 
d č 
c d 
c d c d 
e f e f e f e f 


FIGURE 2 Examples of trees and graphs that are not trees. 


A tree is a connected undirected graph with no simple circuits. 


Because a tree cannot have a simple circuit, a tree cannot contain multiple edges or loops. There- 
fore any tree must be a simple graph. 


Which of the graphs shown in Figure 2 are trees? 


Solution: G, and G, are trees, because both are connected graphs with no simple circuits. G, is 
not a tree because e, b, a, d, e is a simple circuit in this graph. Finally, G, is not a tree because 
it is not connected. < 


Any connected graph that contains no simple circuits is a tree. What about graphs containing 
no simple circuits that are not necessarily connected? These graphs are called forests and have 
the property that each of their connected components is a tree. Figure 3 displays a forest. Note 
that the graph G, in Figure 2 is also a forest. The graph in Figure 3 is a forest of three trees, 
while G, in Figure 2 is a forest of two trees. 

Trees are often defined as undirected graphs with the property that there is a unique simple 
path between every pair of vertices. Theorem 1 shows that this alternative definition is equivalent 
to our definition. 


An undirected graph is a tree if and only if there is a unique simple path between any two of 
its vertices. 


This is one graph with three connected components. 


A 


FIGURE3 Example of a forest. 


Definition 2 
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Proof: First assume that T is a tree. Then T is a connected graph with no simple circuits. Let x 
and y be two vertices of T. Because T is connected, by Theorem 1 of Section 10.4 there is a 
simple path between x and y. Moreover, this path must be unique, for if there were a second 
such path, the path formed by combining the first path from x to y followed by the path from y 
to x obtained by reversing the order of the second path from x to y would form a circuit. This 
implies, using Exercise 59 of Section 10.4, that there is a simple circuit in T. Hence, there is a 
unique simple path between any two vertices of a tree. 

Now assume that there is a unique simple path between any two vertices of a graph T. 
Then T is connected, because there is a path between any two of its vertices. Furthermore, T 
can have no simple circuits. To see that this is true, suppose T had a simple circuit that contained 
the vertices x and y. Then there would be two simple paths between x and y, because the simple 
circuit is made up of a simple path from x to y and a second simple path from y to x. Hence, a 
graph with a unique simple path between any two vertices is a tree. < 


11.1.1 Rooted Trees 


In many applications of trees, a particular vertex of a tree is designated as the root. Once we 
specify a root, we can assign a direction to each edge as follows. Because there is a unique path 
from the root to each vertex of the graph (by Theorem 1), we direct each edge away from the 
root. Thus, a tree together with its root produces a directed graph called a rooted tree. 


A rooted tree is a tree in which one vertex has been designated as the root and every edge is 
directed away from the root. 


Rooted trees can also be defined recursively. Refer to Section 5.3 to see how this can be done. 
We can change an unrooted tree into a rooted tree by choosing any vertex as the root. Note that 
different choices of the root produce different rooted trees. For instance, Figure 4 displays the 
rooted trees formed by designating a to be the root and c to be the root, respectively, in the 
tree T. We usually draw a rooted tree with its root at the top of the graph. The arrows indicating 
the directions of the edges in a rooted tree can be omitted, because the choice of root determines 
the directions of the edges. 

The terminology for trees has botanical and genealogical origins. Suppose that T is a rooted 
tree. If v is a vertex in T other than the root, the parent of v is the unique vertex u such that there 
is a directed edge from u to v (the reader should show that such a vertex is unique). When u is 
the parent of v, v is called a child of u. Vertices with the same parent are called siblings. The 
ancestors of a vertex other than the root are the vertices in the path from the root to this vertex, 
excluding the vertex itself and including the root (that is, its parent, its parent’s parent, and so 
on, until the root is reached). The descendants of a vertex v are those vertices that have v as 
an ancestor. A vertex of a rooted tree is called a leaf if it has no children. Vertices that have 
children are called internal vertices. The root is an internal vertex unless it is the only vertex 
in the graph, in which case it is a leaf. 


T With root a With root c 
č 


c f E 


FIGURE 4 A tree and rooted trees formed by designating two different roots. 
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FIGURE 5 A rooted tree T. FIGURE6 The 
subtree rooted at g. 


If a is a vertex in a tree, the subtree with a as its root is the subgraph of the tree consisting 
of a and its descendants and all edges incident to these descendants. 


In the rooted tree T (with root a) shown in Figure 5, find the parent of c, the children of g, the 
siblings of h, all ancestors of e, all descendants of b, all internal vertices, and all leaves. What 
is the subtree rooted at g? 


Solution: The parent of c is b. The children of g are h, i, and j. The siblings of h are i and j. 
The ancestors of e are c, b, and a. The descendants of b are c, d, and e. The internal vertices 
are a, b, c, g, h, and j. The leaves are d, e, f, i, k, 1, and m. The subtree rooted at g is shown in 
Figure 6. < 


Rooted trees with the property that all of their internal vertices have the same number of 
children are used in many different applications. Later in this chapter we will use such trees to 
study problems involving searching, sorting, and coding. 


A rooted tree is called an m-ary tree if every internal vertex has no more than m children. 
The tree is called a full m-ary tree if every internal vertex has exactly m children. An m-ary 
tree with m = 2 is called a binary tree. 


Are the rooted trees in Figure 7 full m-ary trees for some positive integer m? 


Solution: T, is a full binary tree because each of its internal vertices has two children. T, is a 
full 3-ary tree because each of its internal vertices has three children. In T, each internal vertex 


AN KA BRAS KD 


FIGURE 7 Four rooted trees. 
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has five children, so T; is a full 5-ary tree. T} is not a full m-ary tree for any m because some of 
its internal vertices have two children and others have three children. < 


ORDERED ROOTED TREES An ordered rooted tree is a rooted tree where the children 
of each internal vertex are ordered. Ordered rooted trees are drawn so that the children of each 
internal vertex are shown in order from left to right. Note that a representation of a rooted tree 
in the conventional way determines an ordering for its edges. We will use such orderings of 
edges in drawings without explicitly mentioning that we are considering a rooted tree to be 
ordered. 

In an ordered binary tree (usually called just a binary tree), if an internal vertex has two 
children, the first child is called the left child and the second child is called the right child. 
The tree rooted at the left child of a vertex is called the left subtree of this vertex, and the tree 
rooted at the right child of a vertex is called the right subtree of the vertex. The reader should 
note that for some applications every vertex of a binary tree, other than the root, is designated 
as a right or a left child of its parent. This is done even when some vertices have only one child. 
We will make such designations whenever it is necessary, but not otherwise. 

Ordered rooted trees can be defined recursively. Binary trees, a type of ordered rooted trees, 
were defined this way in Section 5.3. 


What are the left and right children of d in the binary tree T shown in Figure 8(a) (where the 
order is that implied by the drawing)? What are the left and right subtrees of c? 


Solution: The left child of d is f and the right child is g. We show the left and right subtrees of c 
in Figures 8(b) and 8(c), respectively. < 


<. @—o0 >> 
> 
~ 


m 


(b) (c) 


FIGURE 8 A binary tree T and left and right subtrees of the vertex c. 


Just as in the case of graphs, there is no standard terminology used to describe trees, rooted 
trees, ordered rooted trees, and binary trees. This nonstandard terminology occurs because trees 
are used extensively throughout computer science, which is a relatively young field. The reader 
should carefully check meanings given to terms dealing with trees whenever they occur. 


11.1.2 Trees as Models 


Trees are used as models in such diverse areas as computer science, chemistry, geology, botany, 
and psychology. We will describe a variety of such models based on trees. 
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FIGURE 9 The two isomers of butane. 


Saturated Hydrocarbons and Trees Graphs can be used to represent molecules, where 
atoms are represented by vertices and bonds between them by edges. The English mathemati- 
cian Arthur Cayley discovered trees in 1857 when he was trying to enumerate the isomers of 
compounds of the form C,,H,,,,5, which are called saturated hydrocarbons. (Isomers represent 
compounds with the same chemical formula but different chemical properties.) 

In graph models of saturated hydrocarbons, each carbon atom is represented by a vertex 
of degree 4, and each hydrogen atom is represented by a vertex of degree 1. There are 3n + 2 
vertices in a graph representing a compound of the form C,,H,,,,». The number of edges in such 
a graph is half the sum of the degrees of the vertices. Hence, there are (4n + 2n + 2)/2 = 3n+ 1 
edges in this graph. Because the graph is connected and the number of edges is one less than 
the number of vertices, it must be a tree (see Exercise 15). 

The nonisomorphic trees with n vertices of degree 4 and 2n + 2 of degree 1 represent the 
different isomers of C,,H),,,.. For instance, when n = 4, there are exactly two nonisomorphic 
trees of this type (the reader should verify this). Hence, there are exactly two different isomers 
of C,H). Their structures are displayed in Figure 9. These two isomers are called butane and 
isobutane (also known as i-butane or methylpropane). < 


Representing Organizations The structure of a large organization can be modeled using a 
rooted tree. Each vertex in this tree represents a position in the organization. An edge from 
one vertex to another indicates that the person represented by the initial vertex is the (direct) 
boss of the person represented by the terminal vertex. The graph shown in Figure 10 displays 


ARTHUR CAYLEY (1821-1895) Arthur Cayley, the son of a merchant, displayed his mathematical tal- 
ents at an early age with amazing skill in numerical calculations. Cayley entered Trinity College, Cambridge, 
when he was 17. While in college he developed a passion for reading novels. Cayley excelled at Cambridge 
and was elected to a 3-year appointment as Fellow of Trinity and assistant tutor. During this time Cayley 
began his study of n-dimensional geometry and made a variety of contributions to geometry and to anal- 
ysis. He also developed an interest in mountaineering, which he enjoyed during vacations in Switzer- 
land. Because no position as a mathematician was available to him, Cayley left Cambridge, entering the 
legal profession and gaining admittance to the bar in 1849. Although Cayley limited his legal work to 
be able to continue his mathematics research, he developed a reputation as a legal specialist. During his 
legal career he was able to write more than 300 mathematical papers. In 1863 Cambridge University 


established a new post in mathematics and offered it to Cayley. He took this job, even though it paid less money than he made as a 


lawyer. 
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FIGURE 10 An organizational tree for a computer company. 
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EXAMPLE 8 


such a tree. In the organization represented by this tree, the Director of Hardware Development 
works directly for the Vice President of R&D. The root of this tree is the vertex representing 
the President of the organization. < 


Computer File Systems Files in computer memory can be organized into directories. A direc- 
tory can contain both files and subdirectories. The root directory contains the entire file system. 
Thus, a file system may be represented by a rooted tree, where the root represents the root di- 
rectory, internal vertices represent subdirectories, and leaves represent ordinary files or empty 
directories. One such file system is shown in Figure 11. In this system, the file khr is in the 
directory rje. (Note that links to files where the same file may have more than one pathname can 
lead to circuits in computer file systems.) < 


The root is the root directory / 


i Internal vertices are directories 
ae we kg files 
usr bin tmp 


JIN ZIN | 
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ed nroff vikhr opr  uucp 


printer file 


FIGURE 11 A computer file system. 


Tree-Connected Parallel Processors In Example 17 of Section 10.2 we described several 
interconnection networks for parallel processing. A tree-connected network is another impor- 
tant way to interconnect processors. The graph representing such a network is a complete binary 
tree, that is, a full binary tree where every root is at the same level. Such a network interconnects 
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THEOREM 3 


n = 2* — 1 processors, where k is a positive integer. A processor represented by the vertex v 
that is not a root or a leaf has three two-way connections—one to the processor represented by 
the parent of v and two to the processors represented by the two children of v. The processor 
represented by the root has two two-way connections to the processors represented by its two 
children. A processor represented by a leaf v has a single two-way connection to the parent of v. 
We display a tree-connected network with seven processors in Figure 12. 

We now illustrate how a tree-connected network can be used for parallel computation. In 
particular, we show how the processors in Figure 12 can be used to add eight numbers, using 
three steps. In the first step, we add x, and x, using P4, x; and x, using P5, x; and x, using Pe, 
and x- and xg using P}. In the second step, we add x, + x, and x; + x, using P, and x; + x, and 
X7 +X using P}. Finally, in the third step, we add x, + x, + x3 + x4 and x5 + xe + x7 + xg using 
P,. The three steps used to add eight numbers compares favorably to the seven steps required 
to add eight numbers serially, where the steps are the addition of one number to the sum of the 
previous numbers in the list. < 


11.1.3 Properties of Trees 


We will often need results relating the numbers of edges and vertices of various types in trees. 
A tree with n vertices has n — 1 edges. 


Proof: We will use mathematical induction to prove this theorem. Note that for all the trees here 
we can choose a root and consider the tree rooted. 


BASIS STEP: When n = 1, a tree with n = 1 vertex has no edges. It follows that the theorem is 
true forn = 1. 


INDUCTIVE STEP: The inductive hypothesis states that every tree with k vertices has k — 1 
edges, where k is a positive integer. Suppose that a tree T has k + 1 vertices and that v is a leaf of 
T (which must exist because the tree is finite), and let w be the parent of v. Removing from T the 
vertex v and the edge connecting w to v produces a tree T’ with k vertices, because the resulting 
graph is still connected and has no simple circuits. By the inductive hypothesis, T’ has k — 1 
edges. It follows that T has k edges because it has one more edge than 7’, the edge connecting 
v and w. This completes the inductive step. < 


Recall that a tree is a connected undirected graph with no simple circuits. So, when G is an 
undirected graph with n vertices, Theorem 2 tells us that the two conditions (i) G is connected 
and (ii) G has no simple circuits, imply (iii) G has n — 1 edges. Also, when (i) and (iii) hold, 
then (ii) must also hold, and when (ii) and (iii) hold, (i) must also hold. That is, if G is connected 
and G has n — 1 edges, then G has no simple circuits, so that G is a tree (see Exercise 15(a)), 
and if G has no simple circuits and G has n — 1 edges, then G is connected, and so is a tree (see 
Exercise 15(b)). Consequently, when two of (i), (ii), and (iii) hold, the third condition must also 
hold, and G must be a tree. 


COUNTING VERTICES IN FULL m-ARY TREES The number of vertices in a full m-ary 


tree with a specified number of internal vertices is determined, as Theorem 3 shows. As in 
Theorem 2, we will use n to denote the number of vertices in a tree. 


A full m-ary tree with i internal vertices contains n = mi + 1 vertices. 


THEOREM 4 


EXAMPLE 9 


FIGURE 13 A 
rooted tree. 
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Proof: Every vertex, except the root, is the child of an internal vertex. Because each of the i 
internal vertices has m children, there are mi vertices in the tree other than the root. Therefore, 
the tree contains n = mi + | vertices. < 


Suppose that T is a full m-ary tree. Let i be the number of internal vertices and / the number 
of leaves in this tree. Once one of n, i, and / is known, the other two quantities are determined. 
Theorem 4 explains how to find the other two quantities from the one that is known. 


A full m-ary tree with 


(i) n vertices has i = (n — 1)/m internal vertices and / = [(m — 1)n + 1]/m leaves, 
(ii) i internal vertices has n = mi + 1 vertices and / = (m — 1)i + 1 leaves, 
(iii) [leaves has n = (ml — 1)/(m — 1) vertices and i = (l — 1)/(m — 1) internal vertices. 


Proof: Let n represent the number of vertices, i the number of internal vertices, and / the number 
of leaves. The three parts of the theorem can all be proved using the equality given in Theorem 3, 
that is, n = mi + 1, together with the equality n = l + i, which is true because each vertex is 
either a leaf or an internal vertex. We will prove part (i) here. The proofs of parts (ii) and (iii) 
are left as exercises for the reader. 

Solving for i in n=mi+1 gives i=(n-1)/m. Then inserting this expres- 
sion for i into the equation n=/+i shows that /=n-i=n-(n-1)/m = 
[(m— 1)n + 1] /m. 


Example 9 illustrates how Theorem 4 can be used. 


Suppose that someone starts a chain letter. Each person who receives the letter is asked to send 
it on to four other people. Some people do this, but others do not send any letters. How many 
people have seen the letter, including the first person, if no one receives more than one letter 
and if the chain letter ends after there have been 100 people who read it but did not send it out? 
How many people sent out the letter? 


Solution: The chain letter can be represented using a 4-ary tree. The internal vertices correspond 
to people who sent out the letter, and the leaves correspond to people who did not send it out. 
Because 100 people did not send out the letter, the number of leaves in this rooted tree is / = 100. 
Hence, part (iii) of Theorem 4 shows that the number of people who have seen the letter is 
n= (4-100 — 1)/(4 — 1) = 133. Also, the number of internal vertices is 133 — 100 = 33, so 
33 people sent out the letter. < 


BALANCED m-ARY TREES | It is often desirable to use rooted trees that are “balanced” so 
that the subtrees at each vertex contain paths of approximately the same length. Some definitions 
will make this concept clear. The level of a vertex v in a rooted tree is the length of the unique 
path from the root to this vertex. The level of the root is defined to be zero. The height of a 
rooted tree is the maximum of the levels of vertices. In other words, the height of a rooted tree 
is the length of the longest path from the root to any vertex. 


Find the level of each vertex in the rooted tree shown in Figure 13. What is the height of this 
tree? 


Solution: The root a is at level 0. Vertices b, j, and k are at level 1. Vertices c, e, f, and l are at 
level 2. Vertices d, g, i, m, and n are at level 3. Finally, vertex A is at level 4. Because the largest 
level of any vertex is 4, this tree has height 4. < 
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A rooted m-ary tree of height h is balanced if all leaves are at levels h or h — 1. 


Which of the rooted trees shown in Figure 14 are balanced? 


FIGURE 15 Some rooted trees. 


THEOREM 5 


Solution: T, is balanced, because all its leaves are at levels 3 and 4. However, T, is not balanced, 
because it has leaves at levels 2, 3, and 4. Finally, 7, is balanced, because all its leaves are at 
level 3. < 


A BOUND FOR THE NUMBER OF LEAVES IN AN m-ARY TREE It is often useful to have 
an upper bound for the number of leaves in an m-ary tree. Theorem 5 provides such a bound in 
terms of the height of the m-ary tree. 


There are at most m” leaves in an m-ary tree of height A. 


Proof: The proof uses mathematical induction on the height. First, consider m-ary trees of 
height 1. These trees consist of a root with no more than m children, each of which is a leaf. 
Hence, there are no more than m! = m leaves in an m-ary tree of height 1. This is the basis step 
of the inductive argument. 

Now assume that the result is true for all m-ary trees of height less than h; this is the inductive 
hypothesis. Let T be an m-ary tree of height h. The leaves of T are the leaves of the subtrees 
of T obtained by deleting the edges from the root to each of the vertices at level 1, as shown in 
Figure 15. 

Each of these subtrees has height less than or equal to h — 1. So by the inductive hypothesis, 
each of these rooted trees has at most m—! leaves. Because there are at most m such subtrees, 
each with a maximum of m”! leaves, there are at most m - m"! = m” leaves in the rooted tree. 
This finishes the inductive argument. < 


lst 2nd 3rd (m — 1)st mth 
subtree subtree subtree subtree subtree 
of height of height of height _ of height of height 
<h-1 <h-1 <h-1 <h-1 <h-1 


FIGURE 16 The inductive step of the proof. 
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COROLLARY 1 If an mz-ary tree of height h has / leaves, then h > [log,,,/]. If the m-ary tree is full and bal- 
anced, then h = [log,,, l]. (We are using the ceiling function here. Recall that [x] is the small- 
est integer greater than or equal to x.) 


Proof: We know that | < m” from Theorem 5. Taking logarithms to the base m shows that 
log, l < h. Because A is an integer, we have h > [log,, /]. Now suppose that the tree is balanced. 
Then each leaf is at level A or h — 1, and because the height is A, there is at least one leaf at level 
h. It follows that there must be more than m”! leaves (see Exercise 30). Because l < m”, we 
have m”! <1 < m”. Taking logarithms to the base m in this inequality gives h — 1 < log,, 1 < h. 
Hence, h = [log,, I]. 


Exercises 
1. Which of these graphs are trees? 3. Answer these questions about the rooted tree illustrated. 
a) b) a 


N 


c) 


=a 
w 


ES 
ces 


A 


a) Which vertex is the root? 

b) Which vertices are internal? 
2. Which of these graphs are trees? 
b) 


c) Which vertices are leaves? 
a) d) Which vertices are children of j? 
e) Which vertex is the parent of h? 
f) Which vertices are siblings of o? 
g) Which vertices are ancestors of m? 
c) 


h) Which vertices are descendants of b? 


. Answer the same questions as listed in Exercise 3 for the 
rooted tree illustrated. 


e) 


— 
w 
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10. 


11. 


*12. 


* 13. 


*14, 


LF * 15, 


16. 


17. 
18. 


19. 


20. 


21. 
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. Is the rooted tree in Exercise 3 a full m-ary tree for some 


positive integer m? 


. Is the rooted tree in Exercise 4 a full m-ary tree for some 


positive integer m? 


. What is the level of each vertex of the rooted tree in Ex- 


ercise 3? 


. What is the level of each vertex of the rooted tree in Ex- 


ercise 4? 


. Draw the subtree of the tree in Exercise 3 that is rooted 


at 

a) a. b) c. c) e. 

Draw the subtree of the tree in Exercise 4 that is rooted 

at 

a) a. b) c. c) e. 

a) How many nonisomorphic unrooted trees are there 
with three vertices? 

b) How many nonisomorphic rooted trees are there 
with three vertices (using isomorphism for directed 
graphs)? 

a) How many nonisomorphic unrooted trees are there 
with four vertices? 

b) How many nonisomorphic rooted trees are there 
with four vertices (using isomorphism for directed 
graphs)? 

a) How many nonisomorphic unrooted trees are there 
with five vertices? 

b) How many nonisomorphic rooted trees are there with 
five vertices (using isomorphism for directed graphs)? 


Show that a simple graph is a tree if and only if it is 
connected but the deletion of any of its edges produces 
a graph that is not connected. 


Let G be a simple graph with n vertices. Show that 

a) G is a tree if and only if it is connected and has n — 1 
edges. 

b) G is a tree if and only if G has no simple circuits and 
has n — 1 edges. [Hint: To show that G is connected 
if it has no simple circuits and n — 1 edges, show that 
G cannot have more than one connected component. ] 


Which complete bipartite graphs K,,, ,,, where m and n are 


positive integers, are trees? 


m,n? 


How many edges does a tree with 10,000 vertices have? 


How many vertices does a full 5-ary tree with 100 inter- 
nal vertices have? 


How many edges does a full binary tree with 1000 inter- 
nal vertices have? 


How many leaves does a full 3-ary tree with 100 vertices 
have? 


Suppose 1000 people enter a chess tournament. Use a 
rooted tree model of the tournament to determine how 
many games must be played to determine a champion, if 
a player is eliminated after one loss and games are played 
until only one entrant has not lost. (Assume there are no 
ties.) 


22. 


23. 


*24, 


* 25. 


* 26. 


A chain letter starts when a person sends a letter to five 
others. Each person who receives the letter either sends 
it to five other people who have never received it or does 
not send it to anyone. Suppose that 10,000 people send 
out the letter before the chain ends and that no one re- 
ceives more than one letter. How many people receive 
the letter, and how many do not send it out? 

A chain letter starts with a person sending a letter out 
to 10 others. Each person is asked to send the letter out 
to 10 others, and each letter contains a list of the previ- 
ous six people in the chain. Unless there are fewer than 
six names in the list, each person sends one dollar to the 
first person in this list, removes the name of this person 
from the list, moves up each of the other five names one 
position, and inserts his or her name at the end of this 
list. If no person breaks the chain and no one receives 
more than one letter, how much money will a person in 
the chain ultimately receive? 

Either draw a full m-ary tree with 76 leaves and height 3, 
where m is a positive integer, or show that no such tree 
exists. 

Either draw a full m-ary tree with 84 leaves and height 3, 
where m is a positive integer, or show that no such tree 
exists. 

A full m-ary tree T has 81 leaves and height 4. 


a) Give the upper and lower bounds for m. 
b) What is m if T is also balanced? 


A complete m-ary tree is a full m-ary tree in which every leaf 
is at the same level. 


27. 


28. 


29. 


33. 


34. 


Construct a complete binary tree of height 4 and a com- 
plete 3-ary tree of height 3. 

How many vertices and how many leaves does a complete 
m-ary tree of height h have? 

Prove 


a) part (ii) of Theorem 4. 
b) part (iii) of Theorem 4. 


. Show that a full m-ary balanced tree of height h has more 


than m*’~! leaves. 


. How many edges are there in a forest of t trees containing 


a total of n vertices? 


. Explain how a tree can be used to represent the table of 


contents of a book organized into chapters, where each 
chapter is organized into sections, and each section is or- 
ganized into subsections. 

How many different isomers do these saturated hydro- 
carbons have? 


a) C3Hg b) C;H;2 c) C6H4 
What does each of these represent in a tree that represents 
the structure of an organization? 


a) the parent of a vertex 

b) a child of a vertex 

c) a sibling of a vertex 

d) the ancestors of a vertex 

e) the descendants of a vertex 
f) the level of a vertex 

g) the height of the tree 


35. Answer the same questions as those given in Exercise 34 41. 
for a rooted tree representing a computer file system. 
36. a) Draw the complete binary tree with 15 vertices that 
represents a tree-connected network of 15 processors. 
b) Show how 16 numbers can be added using the 15 pro- 
cessors in part (a) using four steps. 


37. Let n be a power of 2. Show that n numbers can be added 42 
in logn steps using a tree-connected network of n -— 1 
processors. 

*38. A labeled tree is a tree where each vertex is assigned a 
label. Two labeled trees are considered isomorphic when *43 


there is an isomorphism between them that preserves the 
labels of vertices. How many nonisomorphic trees are 
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. Show that a center should be chosen as the root to pro- 


duce a rooted tree of minimal height from an unrooted 
tree. 


. Show that a tree has either one center or two centers that 


are adjacent. 


there with three vertices labeled with different integers 44. Show that every tree can be colored using two colors. 

from the set {1, 2, 3}? How many nonisomorphic trees The rooted Fibonacci trees T, are defined recursively in the 

are there with four vertices labeled with different integers following way. T, and T, are both the rooted tree consisting 

from the set {1, 2, 3, 4}? of a single vertex, and for n = 3,4, ..., the rooted tree T, is 
The eccentricity of a vertex in an unrooted tree is the length constructed from a root with 7,,_, as its left subtree and T,,_, 
of the longest simple path beginning at this vertex. A vertex is as its right subtree. 


called a center if no vertex in the tree has smaller eccentricity 
than this vertex. In Exercises 39-41 find every vertex that is 45. 
a center in the given tree. 

c * 46. 
39. a 


47. 


L 48. 


Applications of Trees 


Draw the first seven rooted Fibonacci trees. 


How many vertices, leaves, and internal vertices does the 
rooted Fibonacci tree T, have, where n is a positive inte- 
ger? What is its height? 


What is wrong with the following “proof” using mathe- 
matical induction of the statement that every tree with n 
vertices has a path of length n — 1. Basis step: Every tree 
with one vertex clearly has a path of length 0. Inductive 
step: Assume that a tree with n vertices has a path of 
length n — 1, which has u as its terminal vertex. Add a 
vertex v and the edge from u to v. The resulting tree has 
n + 1 vertices and has a path of length n. This completes 
the inductive step. 


Show that the average depth of a leaf in a binary tree 
with n vertices is Q(log n). 


11.2.1 Introduction 


We will discuss three problems that can be studied using trees. The first problem is: How should 
items in a list be stored so that an item can be easily located? The second problem is: What series 
of decisions should be made to find an object with a certain property in a collection of objects 
of a certain type? The third problem is: How should a set of characters be efficiently coded by 


bit strings? 


11.2.2 Binary Search Trees 


Links » Searching for items in a list is one of the most important tasks that arises in computer science. 
Our primary goal is to implement a searching algorithm that finds items efficiently when the 
items are totally ordered. This can be accomplished through the use of a binary search tree, 
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which is a binary tree in which each child of a vertex is designated as a right or left child, no 
vertex has more than one right child or left child, and each vertex is labeled with a key, which 
is one of the items. Furthermore, vertices are assigned keys so that the key of a vertex is both 
larger than the keys of all vertices in its left subtree and smaller than the keys of all vertices in 
its right subtree. 

This recursive procedure is used to form the binary search tree for a list of items. Start with 
a tree containing just one vertex, namely, the root. The first item in the list is assigned as the 
key of the root. To add a new item, first compare it with the keys of vertices already in the tree, 
starting at the root and moving to the left if the item is less than the key of the respective vertex 
if this vertex has a left child, or moving to the right if the item is greater than the key of the 
respective vertex if this vertex has a right child. When the item is less than the respective vertex 
and this vertex has no left child, then a new vertex with this item as its key is inserted as a 
new left child. Similarly, when the item is greater than the respective vertex and this vertex has 
no right child, then a new vertex with this item as its key is inserted as a new right child. We 
illustrate this procedure with Example 1. 


Form a binary search tree for the words mathematics, physics, geography, zoology, meteorology, 
geology, psychology, and chemistry (using alphabetical order). 


Solution: Figure 1 displays the steps used to construct this binary search tree. The word mathe- 
matics is the key of the root. Because physics comes after mathematics (in alphabetical order), 
add a right child of the root with key physics. Because geography comes before mathemat- 
ics, add a left child of the root with key geography. Next, add a right child of the vertex with 
key physics, and assign it the key zoology, because zoology comes after mathematics and after 
physics. Similarly, add a left child of the vertex with key physics and assign this new vertex the 
key meteorology. Add a right child of the vertex with key geography and assign this new vertex 
the key geology. Add a left child of the vertex with key zoology and assign it the key psychology. 
Add a left child of the vertex with key geography and assign it the key chemistry. (The reader 
should work through all the comparisons needed at each step.) < 


Once we have a binary search tree, we need a way to locate items in the binary search tree, 
as well as a way to add new items. Algorithm 1, an insertion algorithm, actually does both of 


mathematics mathematics mathematics mathematics 
e 
physics 
physics geography physics geography 
zoology 
zoology > mathematics 
physics > mathematics geography < mathematics zoology > physics 
mathematics mathematics mathematics mathematics 
geography geography geography 
hysics `~ hysics = hysics ~ i 
geography EEX as Pay eae physics 
geology A zoology geology zoology zoology 
meteorology zoology meteorology meteorology chemistry r 
psychology meteorology psychology 
psychology > mathematics 
meteorology > mathematics geology < mathematics psychology > physics chemistry < mathematics 
meteorology < physics geology > geography psychology < zoology chemistry < geography 


FIGURE 1 Constructing a binary search tree. 
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these tasks, even though it may appear that it is only designed to add vertices to a binary search 
tree. That is, Algorithm 1 is a procedure that locates an item x in a binary search tree if it is 
present, and adds a new vertex with x as its key if x is not present. In the pseudocode, v is the 
vertex currently under examination and /abel(v) represents the key of this vertex. The algorithm 
begins by examining the root. If x equals the key of v, then the algorithm has found the location 
of x and terminates; if x is less than the key of v, we move to the left child of v and repeat the 
procedure; and if x is greater than the key of v, we move to the right child of v and repeat the 
procedure. If at any step we attempt to move to a child that is not present, we know that x is not 
present in the tree, and we add a new vertex as this child with x as its key. 


ALGORITHM 1 Locating an Item in or Adding an Item to a Binary Search Tree. 


procedure insertion(T: binary search tree, 
x: item) 
v := root of T 
{a vertex not present in T has the value null } 
while v 4 null and label(v) # x 
if x < label(v) then 
if left child of v 4 null then v := left child of v 
else add new vertex as a left child of v and set v := null 
else 
if right child of v 4 null then v := right child of v 
else add new vertex as a right child of v and set v := null 
if root of T = null then add a vertex v to the tree and label it with x 
else if v is null or /abel(v) + x then label new vertex with x and let v be this new vertex 
return v {v = location of x} 


Example 2 illustrates the use of Algorithm 1 to insert a new item into a binary search tree. 


Use Algorithm 1 to insert the word oceanography into the binary search tree in Example 1. 


Solution: Algorithm 1 begins with v, the vertex under examination, equal to the root of T, 
so label(v) = mathematics. Because v # null and label(v) = mathematics < oceanography, we 
next examine the right child of the root. This right child exists, so we set v, the vertex un- 
der examination, to be this right child. At this step we have v # null and label(v) = physics > 
oceanography, so we examine the left child of v. This left child exists, so we set v, the ver- 
tex under examination, to this left child. At this step, we also have v 4 null and label(v) = 
metereology < oceanography, so we try to examine the right child of v. However, this right 
child does not exist, so we add a new vertex as the right child of v (which at this point is the 
vertex with the key metereology) and we set v := null. We now exit the while loop because 
v = null. Because the root of T is not null and v = null, we use the else if statement at the end 
of the algorithm to label our new vertex with the key oceanography. < 


We will now determine the computational complexity of this procedure. Suppose we have 
a binary search tree T for a list of n items. We can form a full binary tree U from T by adding 
unlabeled vertices whenever necessary so that every vertex with a key has two children. This is 
illustrated in Figure 2. Once we have done this, we can easily locate or add a new item as a key 
without adding a vertex. 

The most comparisons needed to add a new item is the length of the longest path in U from 
the root to a leaf. The internal vertices of U are the vertices of T. It follows that U has n internal 
vertices. We can now use part (ii) of Theorem 4 in Section 11.1 to conclude that U has n + 1 
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FIGURE 2 Adding unlabeled vertices to make a binary search tree full. 


leaves. Using Corollary 1 of Section 11.1, we see that the height of U is greater than or equal to 
h = [log,(n + 1)]. So, there must exist an item that requires at least [log,(m + 1)] comparisons 
to be added to the tree. Note that if U is balanced, its height is [log,(” + 1)] (by Corollary 1 
of Section 11.1). Thus, if a binary search tree is balanced, locating or adding an item requires 
no more than [log,(n + 1)] comparisons. A binary search tree can become unbalanced as items 
are added to it. Because balanced binary search trees give optimal worst-case complexity for 
binary searching, algorithms have been devised that rebalance binary search trees as items are 
added. The interested reader can consult references on data structures for the description of such 
algorithms. 


11.2.3 Decision Trees 


Rooted trees can be used to model problems in which a series of decisions leads to a solution. 
For instance, a binary search tree can be used to locate items based on a series of comparisons, 
where each comparison tells us whether we have located the item, or whether we should go 
right or left in a subtree. A rooted tree in which each internal vertex corresponds to a decision, 
with a subtree at these vertices for each possible outcome of the decision, is called a decision 
tree. The possible solutions of the problem correspond to the paths to the leaves of this rooted 
tree. Example 3 illustrates an application of decision trees. 


Suppose there are seven coins, all with the same weight, and a counterfeit coin that weighs less 
than the others. How many weighings are necessary using a balance scale to determine which 
of the eight coins is the counterfeit one? Give an algorithm for finding this counterfeit coin. 


Solution: There are three possibilities for each weighing on a balance scale. The two pans can 
have equal weight, the first pan can be heavier, or the second pan can be heavier. Consequently, 
the decision tree for the sequence of weighings is a 3-ary tree. There are at least eight leaves in 
the decision tree because there are eight possible outcomes (because each of the eight coins can 
be the counterfeit lighter coin), and each possible outcome must be represented by at least one 
leaf. The largest number of weighings needed to determine the counterfeit coin is the height of 
the decision tree. From Corollary 1 of Section 11.1 it follows that the height of the decision tree 
is at least [log, 8] = 2. Hence, at least two weighings are needed. 

It is possible to determine the counterfeit coin using two weighings. The decision tree that 
illustrates how this is done is shown in Figure 3. < 
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FIGURE 3 A decision tree for locating a counterfeit coin. The counterfeit coin is shown in color 


below each final weighing. 


EXAMPLE 4 


THE COMPLEXITY OF COMPARISON-BASED SORTING ALGORITHMS Many differ- 
ent sorting algorithms have been developed. To decide whether a particular sorting algorithm 
is efficient, its complexity is determined. Using decision trees as models, a lower bound for the 
worst-case complexity of sorting algorithms that are based on binary comparisons can be found. 

We can use decision trees to model sorting algorithms and to determine an estimate for the 
worst-case complexity of these algorithms. Note that given n elements, there are n! possible 
orderings of these elements, because each of the n! permutations of these elements can be the 
correct order. The sorting algorithms studied in this book, and most commonly used sorting 
algorithms, are based on binary comparisons, that is, the comparison of two elements at a time. 
The result of each such comparison narrows down the set of possible orderings. Thus, a sorting 
algorithm based on binary comparisons can be represented by a binary decision tree in which 
each internal vertex represents a comparison of two elements. Each leaf represents one of the n! 
permutations of n elements. 


We display in Figure 4 a decision tree that orders the elements of the list a, b, c. < 


FIGURE 4 A decision tree for sorting three distinct elements. 


The complexity of a sort based on binary comparisons is measured in terms of the number 
of such comparisons used. The largest number of binary comparisons ever needed to sort a list 
with n elements gives the worst-case performance of the algorithm. The most comparisons used 
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equals the longest path length in the decision tree representing the sorting procedure. In other 
words, the largest number of comparisons ever needed is equal to the height of the decision 
tree. Because the height of a binary tree with n! leaves is at least [log, n!] (using Corollary 1 
in Section 11.1), at least [log, n!] comparisons are needed, as stated in Theorem 1. 


A sorting algorithm based on binary comparisons requires at least [log, n!] comparisons. 


We can use Theorem | to provide a big-Omega estimate for the number of comparisons used 
by a sorting algorithm based on binary comparison. We need only note that by Exercise 74 in 
Section 3.2 we know that [logn!] is O(n logn), one of the commonly used reference functions 
for the computational complexity of algorithms. Corollary 1 is a consequence of this estimate. 


The number of comparisons used by a sorting algorithm to sort n elements based on binary 
comparisons is Q(n log n). 


A consequence of Corollary 1 is that a sorting algorithm based on binary comparisons that 
uses O(n log n) comparisons, in the worst case, to sort n elements is optimal, in the sense that no 
other such algorithm has better worst-case complexity. Note that by Theorem 1 in Section 5.4 
we see that the merge sort algorithm is optimal in this sense. 

We can also establish a similar result for the average-case complexity of sorting algorithms. 
The average number of comparisons used by a sorting algorithm based on binary comparisons 
is the average depth of a leaf in the decision tree representing the sorting algorithm. By Exer- 
cise 48 in Section 11.1 we know that the average depth of a leaf in a binary tree with N vertices 
is Qdog N). We obtain the following estimate when we let N = n! and note that a function that 
is Q(log n!) is also Q(n logn) because logn! is O(n logn). 


The average number of comparisons used by a sorting algorithm to sort n elements based on 
binary comparisons is Q(n logn). 


11.2.4 Prefix Codes 


Consider the problem of using bit strings to encode the letters of the English alphabet (where 
no distinction is made between lowercase and uppercase letters). We can represent each letter 
with a bit string of length five, because there are only 26 letters and there are 32 bit strings of 
length five. The total number of bits used to encode data is five times the number of characters 
in the text when each character is encoded with five bits. Is it possible to find a coding scheme 
of these letters such that, when data are coded, fewer bits are used? We can save memory and 
reduce transmittal time if this can be done. 

Consider using bit strings of different lengths to encode letters. Letters that occur more 
frequently should be encoded using short bit strings, and longer bit strings should be used to 
encode rarely occurring letters. When letters are encoded using varying numbers of bits, some 
method must be used to determine where the bits for each character start and end. For instance, 
if e were encoded with 0, a with 1, and ¢ with 01, then the bit string 0101 could correspond to 
eat, tea, eaea, or tt. 

One way to ensure that no bit string corresponds to more than one sequence of letters is to 
encode letters so that the bit string for a letter never occurs as the first part of the bit string for 
another letter. Codes with this property are called prefix codes. For instance, the encoding of e 


FIGURE5 A 
binary tree with a 
prefix code. 
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as 0, a as 10, and tas 11 is a prefix code. A word can be recovered from the unique bit string that 
encodes its letters. For example, the string 10110 is the encoding of ate. To see this, note that 
the initial 1 does not represent a character, but 10 does represent a (and could not be the first 
part of the bit string of another letter). Then, the next 1 does not represent a character, but 11 
does represent t. The final bit, 0, represents e. 

A prefix code can be represented using a binary tree, where the characters are the labels of 
the leaves in the tree. The edges of the tree are labeled so that an edge leading to a left child is 
assigned a 0 and an edge leading to a right child is assigned a 1. The bit string used to encode 
a character is the sequence of labels of the edges in the unique path from the root to the leaf 
that has this character as its label. For instance, the tree in Figure 5 represents the encoding of 
e by 0, a by 10, t by 110, n by 1110, and s by 1111. 

The tree representing a code can be used to decode a bit string. For instance, consider the 
word encoded by 11111011100 using the code in Figure 5. This bit string can be decoded by 
starting at the root, using the sequence of bits to form a path that stops when a leaf is reached. 
Each 0 bit takes the path down the edge leading to the left child of the last vertex in the path, 
and each 1 bit corresponds to the right child of this vertex. Consequently, the initial 1111 cor- 
responds to the path starting at the root, going right four times, leading to a leaf in the graph 
that has s as its label, because the string 1111 is the code for s. Continuing with the fifth bit, we 
reach a leaf next after going right then left, when the vertex labeled with a, which is encoded 
by 10, is visited. Starting with the seventh bit, we reach a leaf next after going right three times 
and then left, when the vertex labeled with n, which is encoded by 1110, is visited. Finally, the 
last bit, 0, leads to the leaf that is labeled with e. Therefore, the original word is sane. 

We can construct a prefix code from any binary tree where the left edge at each internal 
vertex is labeled by 0 and the right edge by a 1 and where the leaves are labeled by characters. 
Characters are encoded with the bit string constructed using the labels of the edges in the unique 
path from the root to the leaves. 


HUFFMAN CODING We now introduce an algorithm that takes as input the frequencies 
(which are the probabilities of occurrences) of symbols in a string and produces as output a 
prefix code that encodes the string using the fewest possible bits, among all possible binary 
prefix codes for these symbols. This algorithm, known as Huffman coding, was developed by 
David Huffman in a term paper he wrote in 1951 while a graduate student at MIT. (Note that 
this algorithm assumes that we already know how many times each symbol occurs in the string, 
so we can compute the frequency of each symbol by dividing the number of times this symbol 
occurs by the length of the string.) Huffman coding is a fundamental algorithm in data com- 
pression, the subject devoted to reducing the number of bits required to represent information. 
Huffman coding is extensively used to compress bit strings representing text and it also plays 
an important role in compressing audio and image files. 


DAVID A. HUFFMAN (1925-1999) David Huffman grew up in Ohio. At the age of 18 he received his B.S. 
in electrical engineering from The Ohio State University. Afterward he served in the U.S. Navy as a radar 
maintenance officer on a destroyer that had the mission of clearing mines in Asian waters after World War II. 
Later, he earned his M.S. from Ohio State and his Ph.D. in electrical engineering from MIT. Huffman joined 
the MIT faculty in 1953, where he remained until 1967, when he became the founding member of the computer 
science department at the University of California at Santa Cruz. He played an important role in developing 
this department and spent the remainder of his career there, retiring in 1994. 

Huffman is noted for his contributions to information theory and coding, signal designs for radar and for 
communications, and design procedures for asynchronous logical circuits. His work on surfaces with zero curva- 
ture led him to develop original techniques for folding paper and vinyl into unusual shapes considered works of 


art by many and publicly displayed in several exhibits. However, Huffman is best known for his development of what is now called 
Huffman coding, a result of a term paper he wrote during his graduate work at MIT. 

Huffman enjoyed exploring the outdoors, hiking, and traveling extensively. He became certified as a scuba diver when he was 
in his late 60s. He kept poisonous snakes as pets. 
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Algorithm 2 presents the Huffman coding algorithm. Given symbols and their frequencies, 
our goal is to construct a rooted binary tree where the symbols are the labels of the leaves. The 
algorithm begins with a forest of trees each consisting of one vertex, where each vertex has a 
symbol as its label and where the weight of this vertex equals the frequency of the symbol that 
is its label. At each step, we combine two trees having the least total weight into a single tree 
by introducing a new root and placing the tree with larger weight as its left subtree and the tree 
with smaller weight as its right subtree. Furthermore, we assign the sum of the weights of the 
two subtrees of this tree as the total weight of the tree. (Although procedures for breaking ties by 
choosing between trees with equal weights can be specified, we will not specify such procedures 
here.) The algorithm is finished when it has constructed a tree, that is, when the forest is reduced 
to a single tree. 


ALGORITHM 2 Huffman Coding. 


procedure Huffman(C: symbols a; with frequencies w,, i = 1, ..., n) 
F := forest of n rooted trees, each consisting of the single vertex a; and assigned weight w; 
while F is not a tree 
Replace the rooted trees T and T’ of least weights from F with w(T) > w(T’) with a tree 
having a new root that has T as its left subtree and T” as its right subtree. Label the new 
edge to T with 0 and the new edge to T” with 1. 
Assign w(T) + w(T’) as the weight of the new tree. 
{the Huffman coding for the symbol a; is the concatenation of the labels of the edges in the 
unique path from the root to the vertex a;} 


Example 5 illustrates how Algorithm 2 is used to encode a set of five symbols. 


Use Huffman coding to encode the following symbols with the frequencies listed: A: 0.08, B: 
0.10, C: 0.12, D: 0.15, E: 0.20, F: 0.35. What is the average number of bits used to encode a 
character? 


Solution: Figure 6 displays the steps used to encode these symbols. The encoding produced 
encodes A by 111, B by 110, C by 011, D by 010, E by 10, and F by 00. The average number 
of bits used to encode a symbol using this encoding is 


3- 0.08 +3-0.10+3-0.12+3-0.15+2-0.204+2-0.35 = 2.45. < 


Note that Huffman coding is a greedy algorithm. Replacing the two subtrees with the small- 
est weight at each step leads to an optimal code in the sense that no binary prefix code for these 
symbols can encode these symbols using fewer bits. We leave the proof that Huffman codes are 
optimal as Exercise 32. 

There are many variations of Huffman coding. For example, instead of encoding single 
symbols, we can encode blocks of symbols of a specified length, such as blocks of two symbols. 
Doing so may reduce the number of bits required to encode the string (see Exercise 30). We can 
also use more than two symbols to encode the original symbols in the string (see the preamble 
to Exercise 28). Furthermore, a variation known as adaptive Huffman coding (see [Sa00]) can 
be used when the frequency of each symbol in a string is not known in advance, so that encoding 
is done at the same time the string is being read. 


11.2.5 Game Trees 


Trees can be used to analyze certain types of games such as tic-tac-toe, nim, checkers, and 
chess. In each of these games, two players take turns making moves. Each player knows the 
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FIGURE 6 Huffman coding of symbols in Example 4. 


moves made by the other player and no element of chance enters into the game. We model such 
games using game trees; the vertices of these trees represent the positions that a game can be in 
as it progresses; the edges represent legal moves between these positions. Because game trees 
are usually large, we simplify game trees by representing all symmetric positions of a game by 
the same vertex. However, the same position of a game may be represented by different vertices 
if different sequences of moves lead to this position. The root represents the starting position. 
The usual convention is to represent vertices at even levels by boxes and vertices at odd levels 
by circles. When the game is in a position represented by a vertex at an even level, it is the first 
player’s move; when the game is in a position represented by a vertex at an odd level, it is the 
second player’s move. Game trees may be infinite when the games they represent never end, 
such as games that can enter infinite loops, but for most games there are rules that lead to finite 
game trees. 

The leaves of a game tree represent the final positions of a game. We assign a value to each 
leaf indicating the payoff to the first player if the game terminates in the position represented by 
this leaf. For games that are win-lose, we label a terminal vertex represented by a circle with 
a 1 to indicate a win by the first player and we label a terminal vertex represented by a box with 
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FIGURE7 The game tree for a game of nim. 


a —1 to indicate a win by the second player. For games where draws are allowed, we label a 
terminal vertex corresponding to a draw position with a 0. Note that for win—lose games, we 
have assigned values to terminal vertices so that the larger the value, the better the outcome for 
the first player. 

In Example 6 we display a game tree for a well-known and well-studied game. 


Nim In a version of the game of nim, at the start of a game there are a number of piles of 
stones. Two players take turns making moves; a legal move consists of removing one or more 
stones from one of the piles, without removing all the stones left. A player without a legal move 
loses. (Another way to look at this is that the player removing the last stone loses because the 
position with no piles of stones is not allowed.) The game tree shown in Figure 7 represents this 
version of nim given the starting position where there are three piles of stones containing two, 
two, and one stone each, respectively. We represent each position with an unordered list of the 
number of stones in the different piles (the order of the piles does not matter). The initial move 
by the first player can lead to three possible positions because this player can remove one stone 
from a pile with two stones (leaving three piles containing one, one, and two stones); two stones 
from a pile containing two stones (leaving two piles containing two stones and one stone); or 
one stone from the pile containing one stone (leaving two piles of two stones). When only one 
pile with one stone is left, no legal moves are possible, so such positions are terminal positions. 
Because nim is a win—lose game, we label the terminal vertices with +1 when they represent 
wins for the first player and —1 when they represent wins for the second player. < 


Tic-tac-toe The game tree for tic-tac-toe is extremely large and cannot be drawn here, al- 
though a computer could easily build such a tree. We show a portion of the game tree for tic- 
tac-toe in Figure 8(a). Note that by considering symmetric positions equivalent, we need only 
consider three possible initial moves, as shown in Figure 8(a). We also show a subtree of this 
game tree leading to terminal positions in Figure 8(b), where a player who can win makes a 
winning move. < 


We can recursively define the values of all vertices in a game tree in a way that enables 
us to determine the outcome of this game when both players follow optimal strategies. By a 
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FIGURE 8 Some of the game tree for tic-tac-toe. 


Definition 1 


THEOREM 3 


strategy we mean a set of rules that tells a player how to select moves to win the game. An 
optimal strategy for the first player is a strategy that maximizes the payoff to this player and for 
the second player is a strategy that minimizes this payoff. We now recursively define the value 
of a vertex. 


The value of a vertex in a game tree is defined recursively as: 


(i) the value of a leaf is the payoff to the first player when the game terminates in the 
position represented by this leaf. 

(ii) the value of an internal vertex at an even level is the maximum of the values of its 
children, and the value of an internal vertex at an odd level is the minimum of the 
values of its children. 


The strategy where the first player moves to a position represented by a child with maximum 
value and the second player moves to a position of a child with minimum value is called the 
minmax strategy. We can determine who will win the game when both players follow the 
minmax strategy by calculating the value of the root of the tree; this value is called the value of 
the tree. This is a consequence of Theorem 3. 


The value of a vertex of a game tree tells us the payoff to the first player if both players follow 
the minmax strategy and play starts from the position represented by this vertex. 


Proof: We will use induction to prove this theorem. 


BASIS STEP: If the vertex is a leaf, by definition the value assigned to this vertex is the payoff 
to the first player. 
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EXAMPLE 8 


INDUCTIVE STEP: The inductive hypothesis is the assumption that the values of the children 
of a vertex are the payoffs to the first player, assuming that play starts at each of the positions 
represented by these vertices. We need to consider two cases, when it is the first player’s turn 
and when it is the second player’s turn. 

When it is the first player’s turn, this player follows the minmax strategy and moves to the 
position represented by the child with the largest value. By the inductive hypothesis, this value 
is the payoff to the first player when play starts at the position represented by this child and 
follows the minmax strategy. By the recursive step in the definition of the value of an internal 
vertex at an even level (as the maximum value of its children), the value of this vertex is the 
payoff when play begins at the position represented by this vertex. 

When it is the second player’s turn, this player follows the minmax strategy and moves to 
the position represented by the child with the least value. By the inductive hypothesis, this value 
is the payoff to the first player when play starts at the position represented by this child and both 
players follow the minmax strategy. By the recursive definition of the value of an internal vertex 
at an odd level as the minimum value of its children, the value of this vertex is the payoff when 
play begins at the position represented by this vertex. 


Remark: By extending the proof of Theorem 3, it can be shown that the minmax strategy is the 
optimal strategy for both players. 


Example 8 illustrates how the minmax procedure works. It displays the values assigned to 
the internal vertices in the game tree from Example 6. Note that we can shorten the compu- 
tation required by noting that for win—lose games, once a child of a square vertex with value 
+1 is found, the value of the square vertex is also +1 because +1 is the largest possible payoff. 
Similarly, once a child of a circle vertex with value —1 is found, this is the value of the circle 
vertex also. 


In Example 6 we constructed the game tree for nim with a starting position where there are 
three piles containing two, two, and one stones. In Figure 9 we show the values of the vertices 
of this game tree. The values of the vertices are computed using the values of the leaves and 
working one level up at a time. In the right margin of this figure we indicate whether we use 
the maximum or minimum of the values of the children to find the value of an internal vertex at 


min 


min 


FIGURE 9 Showing the values of vertices in the game of nim. 


Chess programs on 
smartphones can now 
play at the grandmaster 
level. 
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each level. For example, once we have found the values of the three children of the root, which 
are 1, —1, and —1, we find the value of the root by computing max(1, —1, —1) = 1. Because the 
value of the root is 1, it follows that the first player wins when both players follow a minmax 
strategy. < 


Game trees for some well-known games can be extraordinarily large, because these games 
have many different possible moves. For example, the game tree for chess has been estimated to 
have as many as 10!”° vertices! It may be impossible to use Theorem 3 directly to study a game 
because of the size of the game tree. Therefore, various approaches have been devised to help 
determine good strategies and to determine the outcome of such games. One useful technique, 
called alpha-beta pruning, eliminates much computation by pruning portions of the game tree 
that cannot affect the values of ancestor vertices. (For information about alpha-beta pruning, 
consult [Gr90].) Another useful approach is to use evaluation functions, which estimate the 
value of internal vertices in the game tree when it is not feasible to compute these values exactly. 
For example, in the game of tic-tac-toe, as an evaluation function for a position, we may use the 
number of files (rows, columns, and diagonals) containing no Os (used to indicate moves of the 
second player) minus the number of files containing no Xs (used to indicate moves of the first 
player). This evaluation function provides some indication of which player has the advantage 
in the game. Once the values of an evaluation function are inserted, the value of the game can 
be computed following the rules used for the minmax strategy. Computer programs created to 
play chess, such as IBM’s Deep Blue, the first chess-playing computer to win a match against 
a current world champion under regular rules, are based on sophisticated evaluation functions. 
For more information about how computers play chess see [Le91]. 

The material we have studied comes from combinatorial game theory, which deals 
with games where a player knows all previous moves and selects a move before other players 
choose theirs. For more information about combinatorial game theory, consult [AINoWo07], 
[BeCoGu82a, 82b], or [Be04], and the web links for this subject. 


. Build a binary search tree for the words banana, peach, 
apple, pear, coconut, mango, and papaya using alphabet- 
ical order. 

. Build a binary search tree for the words venology, 
phrenology, campanology, ornithology, ichthyology, lim- 
nology, alchemy, and astrology using alphabetical order. 
. How many comparisons are needed to locate or to add 
each of these words in the search tree for Exercise 
1, starting fresh each time? 

b) banana 

d) orange 


a) pear 

c) kumquat 
. How many comparisons are needed to locate or to add 
each of the words in the search tree for Exercise 2, start- 
ing fresh each time? 


*8, 


*Q, 


. How many weighings of a balance scale are needed to 


find a counterfeit coin among four coins if the counter- 
feit coin may be either heavier or lighter than the others? 
Describe an algorithm to find the counterfeit coin using 
this number of weighings. 

How many weighings of a balance scale are needed to 
find a counterfeit coin among eight coins if the counter- 
feit coin is either heavier or lighter than the others? De- 
scribe an algorithm to find the counterfeit coin using this 
number of weighings. 

How many weighings of a balance scale are needed to 
find a counterfeit coin among 12 coins if the counterfeit 
coin is lighter than the others? Describe an algorithm to 
find the lighter coin using this number of weighings. 


* 10. One of four coins may be counterfeit. If it is counterfeit, 
a) palmistry b) etymology it may be lighter or heavier than the others. How many 
c) paleontology d) glaciology weighings are needed, using a balance scale, to determine 
. Using alphabetical order, construct a binary search tree whether there is a counterfeit coin, and if there is, whether 
for the words in the sentence “The quick brown fox jumps it is lighter or heavier than the others? Describe an algo- 
over the lazy dog.” rithm to find the counterfeit coin and determine whether 

. How many weighings of a balance scale are needed to it is lighter or heavier using this number of weighings. 
find a lighter counterfeit coin among four coins? Describe 11. Find the least number of comparisons needed to sort four 


an algorithm to find the lighter coin using this number of 
weighings. 


elements and devise an algorithm that sorts these ele- 
ments using this number of comparisons. 
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* 12. Find the least number of comparisons needed to sort five 
elements and devise an algorithm that sorts these ele- 
ments using this number of comparisons. 


The tournament sort is a sorting algorithm that works by 
building an ordered binary tree. We represent the elements to 
be sorted by vertices that will become the leaves. We build 
up the tree one level at a time as we would construct the tree 
representing the winners of matches in a tournament. Work- 
ing left to right, we compare pairs of consecutive elements, 
adding a parent vertex labeled with the larger of the two ele- 
ments under comparison. We make similar comparisons be- 
tween labels of vertices at each level until we reach the root of 
the tree that is labeled with the largest element. The tree con- 
structed by the tournament sort of 22, 8, 14, 17, 3, 9, 27, 11 
is illustrated in part (a) of the figure. Once the largest element 
has been determined, the leaf with this label is relabeled by 
—coo, which is defined to be less than every element. The labels 
of all vertices on the path from this vertex up to the root of the 
tree are recalculated, as shown in part (b) of the figure. This 
produces the second largest element. This process continues 
until the entire list has been sorted. 


(27) 27 is largest element 


13. Complete the tournament sort of the list 22, 8, 14, 17, 3, 
9, 27, 11. Show the labels of the vertices at each step. 


14. Use the tournament sort to sort the list 17, 4, 1, 5, 13, 10, 
14, 6. 


15. Describe the tournament sort using pseudocode. 


16. Assuming that n, the number of elements to be sorted, 
equals 2" for some positive integer k, determine the num- 
ber of comparisons used by the tournament sort to find 
the largest element of the list using the tournament sort. 

17. How many comparisons does the tournament sort use to 
find the second largest, the third largest, and so on, up to 
the (n — 1)st largest (or second smallest) element? 

18. Show that the tournament sort requires O(n log n) com- 
parisons to sort a list of n elements. [Hint: By inserting 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


the appropriate number of dummy elements defined to be 
smaller than all integers, such as —oo, assume that n = 2k 
for some positive integer k.] 

Which of these codes are prefix codes? 

a) a: 11, e: 00, t: 10, s: O1 

b) a: 0, e: 1, t: 01, s: 001 

c) a: 101, e: 11, t: 001, s: 011, 7: 010 

d) a: 010, e: 11, t: 011, s: 1011, n: 1001, i: 10101 


Construct the binary tree with prefix codes representing 
these coding schemes. 

a) a: 11, e: 0, t: 101, s: 100 

b) a: 1, e: 01, t: 001, s: 0001, n: 00001 

c) a: 1010, e: 0, t: 11, s: 1011, n: 1001, 7: 10001 


What are the codes for a, e, i, k, o, p, and u if the coding 
scheme is represented by this tree? 


Given the coding scheme a: 001, b: 0001, e: 1, r: 0000, 
s: 0100, t: 011, x: 01010, find the word represented by 


a) 01110100011. b) 0001110000. 
c) 0100101010. d) 01100101010. 


Use Huffman coding to encode these symbols with given 
frequencies: a: 0.20, b: 0.10, c: 0.15, d: 0.25, e: 0.30. 
What is the average number of bits required to encode 
a character? 

Use Huffman coding to encode these symbols with given 
frequencies: A: 0.10, B: 0.25, C: 0.05, D: 0.15, E: 0.30, 
F: 0.07, G: 0.08. What is the average number of bits re- 
quired to encode a symbol? 


Construct two different Huffman codes for these symbols 

and frequencies: t: 0.2, u: 0.3, v: 0.2, w: 0.3. 

a) Use Huffman coding to encode these symbols with 
frequencies a: 0.4, b: 0.2, c: 0.2, d: 0.1, e: 0.1 in two 
different ways by breaking ties in the algorithm dif- 
ferently. First, among the trees of minimum weight 
select two trees with the largest number of vertices 
to combine at each stage of the algorithm. Second, 
among the trees of minimum weight select two trees 
with the smallest number of vertices at each stage. 

b) Compute the average number of bits required to en- 
code a symbol with each code and compute the vari- 
ances of this number of bits for each code. Which 
tie-breaking procedure produced the smaller variance 
in the number of bits required to encode a symbol? 


27. 


Construct a Huffman code for the letters of the English al- 
phabet where the frequencies of letters in typical English 
text are as shown in this table. 


Pees aed 
A N 


B 
C 
D 
E 
F 
G 
H 
I 
J 
K 
L 
M 


NKXÍ<SCHaAWOTWO 


Suppose that m is a positive integer with m > 2. An m-ary 
Huffman code for a set of N symbols can be constructed anal- 
ogously to the construction of a binary Huffman code. At the 
initial step, ((N — 1) mod (m — 1)) + 1 trees consisting of a 
single vertex with least weights are combined into a rooted 
tree with these vertices as leaves. At each subsequent step, 
the m trees of least weight are combined into an m-ary tree. 


28. 


29. 


30. 


31. 


*32. 


Describe the m-ary Huffman coding algorithm in 
pseudocode. 


Using the symbols 0, 1, and 2 use ternary (m = 3) 
Huffman coding to encode these letters with the given 
frequencies: A: 0.25, E: 0.30, N: 0.10, R: 0.05, T: 0.12, 
Z: 0.18. 


Consider the three symbols A, B, and C with frequencies 

A: 0.80, B: 0.19, C: 0.01. 

a) Construct a Huffman code for these three symbols. 

b) Form a new set of nine symbols by grouping together 
blocks of two symbols, AA, AB, AC, BA, BB, BC, 
CA, CB, and CC. Construct a Huffman code for these 
nine symbols, assuming that the occurrences of sym- 
bols in the original text are independent. 

c) Compare the average number of bits required to en- 
code text using the Huffman code for the three sym- 
bols in part (a) and the Huffman code for the nine 
blocks of two symbols constructed in part (b). Which 
is more efficient? 

Given n+1 symbols x, Xz, ...,%,,%,4, appearing 1, 

fofa ---- f, times in a symbol string, respectively, where 

Í is the jth Fibonacci number, what is the maximum num- 

ber of bits used to encode a symbol when all possible tie- 

breaking selections are considered at each stage of the 

Huffman coding algorithm? 

Show that Huffman codes are optimal in the sense that 

they represent a string of symbols using the fewest bits 

among all binary prefix codes. 
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37. 


38. 


11.2 Applications of Trees 807 


Draw a game tree for nim if the starting position consists 
of two piles with two and three stones, respectively. When 
drawing the tree represent by the same vertex symmetric 
positions that result from the same move. Find the value 
of each vertex of the game tree. Who wins the game if 
both players follow an optimal strategy? 


Draw a game tree for nim if the starting position con- 
sists of three piles with one, two, and three stones, respec- 
tively. When drawing the tree represent by the same ver- 
tex symmetric positions that result from the same move. 
Find the value of each vertex of the game tree. Who wins 
the game if both players follow an optimal strategy? 


Suppose that we vary the payoff to the winning player in 

the game of nim so that the payoff is n dollars when n is 

the number of legal moves made before a terminal posi- 

tion is reached. Find the payoff to the first player if the 

initial position consists of 

a) two piles with one and three stones, respectively. 

b) two piles with two and four stones, respectively. 

c) three piles with one, two, and three stones, 
respectively. 

Suppose that in a variation of the game of nim we allow a 

player to either remove one or more stones from a pile or 

merge the stones from two piles into one pile as long as at 

least one stone remains. Draw the game tree for this vari- 

ation of nim if the starting position consists of three piles 

containing two, two, and one stone, respectively. Find the 

values of each vertex in the game tree and determine the 

winner if both players follow an optimal strategy. 


Draw the subtree of the game tree for tic-tac-toe begin- 
ning at each of these positions. Determine the value of 
each of these subtrees. 


a) Oo] x|x b x|olx 
xlolo o| x|x 
map 
0) x O d) olx 
olo xlo 
xl fx xlo 


Suppose that the first four moves of a tic-tac-toe game are 
as shown. Does the first player (whose moves are marked 
by Xs) have a strategy that will always win? 


a xļlolx 


b x/o|x 
st 
5 — 

c) eal d) x 
x xlo 
O el; il 
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Show that if a game of nim begins with two piles contain- 
ing the same number of stones, as long as this number is 
at least two, then the second player wins when both play- 
ers follow optimal strategies. 


Show that if a game of nim begins with two piles con- 
taining different numbers of stones, the first player wins 
when both players follow optimal strategies. 


How many children does the root of the game tree for 
checkers have? How many grandchildren does it have? 


How many children does the root of the game tree for 
nim have and how many grandchildren does it have if the 
starting position is 


a) piles with four and five stones, respectively. 
b) piles with two, three, and four stones, respectively. 


Tree Traversal 


43. 


44. 


c) piles with one, two, three, and four stones, respec- 
tively. 
d) piles with two, two, three, three, and five stones, re- 
spectively. 
Draw the game tree for the game of tic-tac-toe for the 
levels corresponding to the first two moves. Assign the 
value of the evaluation function mentioned in the text 
that assigns to a position the number of files containing 
no Os minus the number of files containing no Xs as the 
value of each vertex at this level and compute the value of 
the tree for vertices as if the evaluation function gave the 
correct values for these vertices. 


Use pseudocode to describe an algorithm for determin- 
ing the value of a game tree when both players follow a 
minmax strategy. 


11.3.1 Introduction 


Links > 


Ordered rooted trees are often used to store information. We need procedures for visiting each 
vertex of an ordered rooted tree to access data. We will describe several important algorithms 


for visiting all the vertices of an ordered rooted tree. Ordered rooted trees can also be used 
to represent various types of expressions, such as arithmetic expressions involving numbers, 
variables, and operations. The different listings of the vertices of ordered rooted trees used to 
represent expressions are useful in the evaluation of these expressions. 


11.3.2 Universal Address Systems 


Procedures for traversing all vertices of an ordered rooted tree rely on the orderings of children. 
In ordered rooted trees, the children of an internal vertex are shown from left to right in the 
drawings representing these directed graphs. 

We will describe one way we can totally order the vertices of an ordered rooted tree. To 
produce this ordering, we must first label all the vertices. We do this recursively: 


1. Label the root with the integer 0. Then label its k children (at level 1) from left to right 


with 1, 2,3,...,k. 


2. For each vertex v at level n with label A, label its k, children, as they are drawn from left 


to right, with A.1, A.2, ..., A.k,,. 


Following this procedure, a vertex v at level n, forn > 1, is labeled x,.x,.... 


.X,,, where the unique 


path from the root to v goes through the x, st vertex at level 1, the x,nd vertex at level 2, and so 
on. This labeling is called the universal address system of the ordered rooted tree. 

We can totally order the vertices using the lexicographic ordering of their labels in the uni- 
versal address system. The vertex labeled x,.x,.... .x,, is less than the vertex labeled y,.yy.... Ym 


if there is an 7, 0 < i < n, with x, = y1, x, = Yz 


x; =y, fori=1,2,...,n. 


EXAMPLE 1 


, Xi-1 = Yj and x; < y;; or if n < m and 


We display the labelings of the universal address system next to the vertices in the ordered 


rooted tree shown in Figure 1. The lexicographic ordering of the labelings is 


Extra » 
Examples 


0<1<1.1<12<13<2<3<3.1 < 3.1.1 < 3.1.2 < 3.1.2.1 < 3.1.2.2 


< 3.1.2.3 < 3.1.2.4<3.13<32<4<41<5<51<51.1<52<53 < 


Definition 1 


EXAMPLE 2 
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3.1.2.1 / 3.1.2.3 | 
3.1.2.2 3.1.2.4 


FIGURE 1 The universal address system of an ordered rooted tree. 


11.3.3 Traversal Algorithms 


Procedures for systematically visiting every vertex of an ordered rooted tree are called traversal 
algorithms. We will describe three of the most commonly used such algorithms, preorder 
traversal, inorder traversal, and postorder traversal. Each of these algorithms can be defined 
recursively. We first define preorder traversal. 


Let T be an ordered rooted tree with root r. If T consists only of r, then r is the preorder 
traversal of T. Otherwise, suppose that T}, T), ..., T, are the subtrees at r from left to right 
in T. The preorder traversal begins by visiting r. It continues by traversing 7, in preorder, 
then T, in preorder, and so on, until T, is traversed in preorder. 


The reader should verify that the preorder traversal of an ordered rooted tree gives the same 
ordering of the vertices as the ordering obtained using a universal address system. Figure 2 
indicates how a preorder traversal is carried out. 

Example 2 illustrates preorder traversal. 


In which order does a preorder traversal visit the vertices in the ordered rooted tree T shown in 
Figure 3? 


Solution: The steps of the preorder traversal of T are shown in Figure 4. We traverse T in pre- 
order by first listing the root a, followed by the preorder list of the subtree with root b, the 
preorder list of the subtree with root c (which is just c) and the preorder list of the subtree with 
root d. 

The preorder list of the subtree with root b begins by listing b, then the vertices of the 
subtree with root e in preorder, and then the subtree with root f in preorder (which is just f). 
The preorder list of the subtree with root d begins by listing d, followed by the preorder list of 
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Step 1: Visit r 


Preorder traversal 


Step 2: Step 3: Stepn+ 1: 
Visit T) Visit T> Visit T, 
in preorder in preorder in preorder 
FIGURE 2 Preorder traversal. FIGURE 3. The ordered 
rooted tree T. 
a 


Preorder traversal: Visit root, 
visit subtrees left to right 


m 


a b e j k fce d g lm h i 


e eee e o o o o o ee 

n o p 
a b e j k n o p f c dg lm h i 
e o e © o o o o e © o o o o 


FIGURE 4 The preorder traversal of T. 
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Step 2: Visit r 


Inorder traversal 


Step 1: Step 3: Step n + 1: 
Visit T} in Visit T, in Visit T,, in 
inorder inorder inorder 


FIGURE 5 Inorder traversal. 


the subtree with root g, followed by the subtree with root h (which is just A), followed by the 
subtree with root i (which is just 7). 

The preorder list of the subtree with root e begins by listing e, followed by the preorder 
listing of the subtree with root j (which is just j), followed by the preorder listing of the subtree 
with root k. The preorder listing of the subtree with root g is g followed by J, followed by m. 
The preorder listing of the subtree with root k is k, n, o, p. Consequently, the preorder traversal 
of T is a, b, e, j, k,n, o, p, f, c, d, g, l, m, h, i. < 


We will now define inorder traversal. 


Let T be an ordered rooted tree with root r. If T consists only of r, then r is the inorder traver- 
sal of T. Otherwise, suppose that T,, T3, ..., T, are the subtrees at r from left to right. The 
inorder traversal begins by traversing 7, in inorder, then visiting r. It continues by traversing 
T, in inorder, then 73 in inorder, ..., and finally T, in inorder. 


Figure 5 indicates how inorder traversal is carried out. Example 3 illustrates how inorder 
traversal is carried out for a particular tree. 


In which order does an inorder traversal visit the vertices of the ordered rooted tree T in Figure 3? 


Solution: The steps of the inorder traversal of the ordered rooted tree T are shown in Figure 6. 
The inorder traversal begins with an inorder traversal of the subtree with root b, the root a, the 
inorder listing of the subtree with root c, which is just c, and the inorder listing of the subtree 
with root d. 

The inorder listing of the subtree with root b begins with the inorder listing of the sub- 
tree with root e, the root b, and f. The inorder listing of the subtree with root d begins with 
the inorder listing of the subtree with root g, followed by the root d, followed by h, followed 
by i. 

The inorder listing of the subtree with root e is j, followed by the root e, followed by the 
inorder listing of the subtree with root k. The inorder listing of the subtree with root g is L, g, m. 
The inorder listing of the subtree with root k is n, k, o, p. Consequently, the inorder listing of 
the ordered rooted tree is j, e, n, k, o, p, b, f, a, c, l, g, m, d, h, i. < 


We now define postorder traversal. 
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Definition 3 


EXAMPLE 4 
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Inorder traversal: Visit leftmost 
subtree, visit root, visit other 
subtrees left to right 


h i 
ee 
m d h i 


j esa k o p b fac l gem d hA 


FIGURE 6 The inorder traversal of T. 


Let T be an ordered rooted tree with root r. If T consists only of r, then r is the postorder 
traversal of T. Otherwise, suppose that 7, 75, ..., T, are the subtrees at r from left to right. 
The postorder traversal begins by traversing T, in postorder, then T, in postorder, ..., then T, 
in postorder, and ends by visiting r. 


Figure 7 illustrates how postorder traversal is done. Example 4 illustrates how postorder 
traversal works. 


In which order does a postorder traversal visit the vertices of the ordered rooted tree T shown 
in Figure 3? 


Solution: The steps of the postorder traversal of the ordered rooted tree T are shown in Fig- 
ure 8. The postorder traversal begins with the postorder traversal of the subtree with root b, the 


Step n + 1: Visit r 


Postorder traversal 


Step 1: Step 2: Step n: 
Visit T) Visit T, Visit T, 
in postorder in postorder in postorder 


FIGURE 7 Postorder traversal. 


Postorder traversal: Visit 
subtrees left to right; visit root 


a 
e 
h i d a 
eee 
j k e fbc lm g hida 
e oeeeeeeeeee 
n o p 
j n o p k e f b c Lm g h i a 
e o o o o e e o o o o e 


FIGURE 8 The postorder traversal of T. 
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postorder traversal of the subtree with root c, which is just c, the postorder traversal of the subtree 
with root d, followed by the root a. 

The postorder traversal of the subtree with root b begins with the postorder traversal of the 
subtree with root e, followed by f, followed by the root b. The postorder traversal of the rooted 
tree with root d begins with the postorder traversal of the subtree with root g, followed by h, 
followed by i, followed by the root d. 

The postorder traversal of the subtree with root e begins with j, followed by the postorder 
traversal of the subtree with root k, followed by the root e. The postorder traversal of the subtree 
with root g is l, m, g. The postorder traversal of the subtree with root k is n, o, p, k. Therefore, 
the postorder traversal of T is j, n, o, p, k, e, f, b, c, l, m, g, h, i, d, a. < 


There are easy ways to list the vertices of an ordered rooted tree in preorder, inorder, and 
postorder. To do this, first draw a curve around the ordered rooted tree starting at the root, moving 
along the edges, as shown in the example in Figure 9. We can list the vertices in preorder by 
listing each vertex the first time this curve passes it. We can list the vertices in inorder by listing 
a leaf the first time the curve passes it and listing each internal vertex the second time the curve 
passes it. We can list the vertices in postorder by listing a vertex the last time it is passed on the 
way back up to its parent. When this is done in the rooted tree in Figure 9, it follows that the 
preorder traversal gives a, b, d, h, e, i, j, c, f, g, k, the inorder traversal gives h, d, b, i, e, j, a, f, 
c, k, g, and the postorder traversal gives h, d, i, j, e, b, f, k, 8, c, a. 


FIGURE 9 A shortcut for traversing an ordered 
rooted tree in preorder, inorder, and postorder. 


Algorithms for traversing ordered rooted trees in preorder, inorder, or postorder are most 
easily expressed recursively. 


ALGORITHM 1 Preorder Traversal. 


procedure preorder(T: ordered rooted tree) 
r := root of T 
list r 
for each child c of r from left to right 
T(c) := subtree with c as its root 
preorder(T(c)) 
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ALGORITHM 2 Inorder Traversal. 


procedure inorder(T: ordered rooted tree) 
r := root of T 
if r is a leaf then list r 
else 
l := first child of r from left to right 
T(2) := subtree with / as its root 
inorder(T(1)) 
list r 
for each child c of r except for / from left to right 
T(c) := subtree with c as its root 
inorder(T(c)) 


ALGORITHM 3 Postorder Traversal. 


procedure postorder(T: ordered rooted tree) 
r := root of T 
for each child c of r from left to right 
T(c) := subtree with c as its root 
postorder(T(c)) 
list r 


Both the preorder traversal and the postorder traversal encode the structure of an ordered 
rooted tree when the number of children of each vertex is specified. That is, an ordered rooted 
tree is uniquely determined when we specify a list of vertices generated by a preorder traversal 
or by a postorder traversal of the tree, together with the number of children of each vertex (see 
Exercises 26 and 27). In particular, both a preorder traversal and a postorder traversal encode 
the structure of a full ordered m-ary tree. However, when the number of children of vertices is 
not specified, neither a preorder traversal nor a postorder traversal encodes the structure of an 
ordered rooted tree (see Exercises 28 and 29). 


Uses of Inorder, Preorder, and Postorder Traversals Tree traversals have many applications 
and they play a key role in the implementation of many algorithms. Binary ordered trees can 
be used to represent well-formed expressions of objects and operations. Preorder, postorder, 
and inorder traversals of these trees yield the prefix, postfix, and infix notations of expressions, 
which can then be used in applications. Before we turn our attention to that use of tree traversals, 
we will provide useful guidance about the practical use of tree traversals. 

When efficiency is not an issue, the vertices of a tree can be visited in any order, so long as 
each node is visited exactly once. However, for other applications we may need to visit vertices 
in an order that preserves a specific relationship. Also, when efficiency is important, the most 
efficient traversal method for that application should be used. A general principle for deciding 
the traversal to use is to explore the vertices of interest as soon as possible. Preorder traversal is 
the best choice for applications where internal vertices must be explored before leaves. Preorder 
traversals are also used to make copies of binary search trees. 

An interesting digression is to note that preorder traversal has ancient roots. According 
to Knuth [Kn98], when a king, duke, or earl dies, his title passes to his first son, then to the 
descendants of this son; when none of these are alive, it passes to the second son, and then 
his descendants, and so on. (In more modern times, daughters have also been included in this 
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order.) So, a preorder traversal of the vertices of the relevant family tree produces the order of 
succession to the throne, once deceased members are removed. 

Postorder traversal is the best choice for applications where leaves need to be explored 
before internal vertices. Postorder explores leaves before internal vertices, so it is the best choice 
for deleting a tree because the vertices below the root of a subtree can be removed before the 
root of the subtree. Topological sorting is an example of an algorithm that can be efficiently 
implemented using postorder traversal. An inorder traversal of a binary search tree, discussed 
in Section 11.2, visits the vertices in ascending order of their key values. Such a traversal creates 
a sorted list of the data in a binary tree. 


11.3.4 Infix, Prefix, and Postfix Notation 


We can represent complicated expressions, such as compound propositions, combinations of 
sets, and arithmetic expressions using ordered rooted trees. For instance, consider the repre- 
sentation of an arithmetic expression involving the operators + (addition), — (subtraction), * 
(multiplication), / (division), and ¢ (exponentiation). We will use parentheses to indicate the or- 
der of the operations. An ordered rooted tree can be used to represent such expressions, where 
the internal vertices represent operations, and the leaves represent the variables or numbers. 
Each operation operates on its left and right subtrees (in that order). 


What is the ordered rooted tree that represents the expression ((x + y) 12) + ((x — 4)/3)? 


Solution: The binary tree for this expression can be built from the bottom up. First, a sub- 
tree for the expression x+ y is constructed. Then this is incorporated as part of the larger 
subtree representing (x + y)Î2. Also, a subtree for x — 4 is constructed, and then this is in- 
corporated into a subtree representing (x — 4)/3. Finally the subtrees representing (x + y) Î 2 
and (x-—4)/3 are combined to form the ordered rooted tree representing ((x + y) f 2)+ 
((x — 4)/3). These steps are shown in Figure 10. < 


An inorder traversal of the binary tree representing an expression produces the original 
expression with the elements and operations in the same order as they originally occurred, ex- 
cept for unary operations, which instead immediately follow their operands. For instance, in- 
order traversals of the binary trees in Figure 11, which represent the expressions (x + y)/(x + 3), 
(x + (y/x)) + 3, and x + (y/(x + 3)), all lead to the infix expression x + y/x + 3. To make such 
expressions unambiguous it is necessary to include parentheses in the inorder traversal when- 
ever we encounter an operation. The fully parenthesized expression obtained in this way is said 
to be in infix form. 


A, 
: AN A 
r. 


FIGURE 10 A binary tree representing ((x + y) ¢ 2) + (x — 4)/3). 
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We obtain the prefix form of an expression when we traverse its rooted tree in preorder. 
Expressions written in prefix form are said to be in Polish notation, which is named after the 
Polish logician Jan Lukasiewicz. An expression in prefix notation (where each operation has 
a specified number of operands), is unambiguous, so no parentheses are needed in such an 
expression. The verification of this is left as an exercise for the reader. 


What is the prefix form for ((x + y) f 2) + (x — 4)/3)? 


Solution: We obtain the prefix form for this expression by traversing the binary tree that repre- 
sents it in preorder, shown in Figure 10. This produces + f +xy2/—x43. < 


In the prefix form of an expression, a binary operator, such as +, precedes its two operands. 
Hence, we can evaluate an expression in prefix form by working from right to left. When we 
encounter an operator, we perform the corresponding operation with the two operands imme- 
diately to the right of this operand. Also, whenever an operation is performed, we consider the 
result a new operand. 


What is the value of the prefix expression + — * 235/723 4? 
+ + 
oN FS 
en 
+ $ 7 \ / i“ 
PX IS FS 


FIGURE 11 Rooted trees representing (x + y)/(x + 3), (x + (y/x)) + 3, and 
x +(y/(« + 3)). 


Solution: The steps used to evaluate this expression by working right to left, and performing 
operations using the operands on the right, are shown in Figure 12. The value of this expression 


is 3. < 


We obtain the postfix form of an expression by traversing its binary tree in postorder. Ex- 
pressions written in postfix form are said to be in reverse Polish notation. Expressions in re- 
verse Polish notation are unambiguous, so parentheses are not needed. The verification of this 
is left to the reader. Reverse polish notation was extensively used in electronic calculators in 
the 1970s and 1980s. 


What is the postfix form of the expression ((x + y) f 2) + (a — 4)/3)? 


Solution: The postfix form of the expression is obtained by carrying out a postorder traversal of 
the binary tree for this expression, shown in Figure 10. This produces the postfix expression: x 
yt2tx4—-3/+4. < 


In the postfix form of an expression, a binary operator follows its two operands. So, to 
evaluate an expression from its postfix form, work from left to right, carrying out operations 
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14+2=3 14+3=4 


Value of expression: 3 Value of expression: 4 


FIGURE 12 Evaluating a prefix 
expression. 


FIGURE 13 
expression. 


Evaluating a postfix 


whenever an operator follows two operands. After an operation is carried out, the result of this 
operation becomes a new operand. 


What is the value of the postfix expression 7 2 3 x —4 f 9 3/+? 


Solution: The steps used to evaluate this expression by starting at the left and carrying out 
operations when two operands are followed by an operator are shown in Figure 13. The value 
of this expression is 4. < 


Rooted trees can be used to represent other types of expressions, such as those representing 
compound propositions and combinations of sets. In these examples unary operators, such as 
the negation of a proposition, occur. To represent such operators and their operands, a vertex 
representing the operator and a child of this vertex representing the operand are used. 


Find the ordered rooted tree representing the compound proposition (~(p A q)) e (Ap V 7q). 
Then use this rooted tree to find the prefix, postfix, and infix forms of this expression. 


JAN ŁUKASIEWICZ (1878-1956) Jan Lukasiewicz was born into a Polish-speaking family in Lvov. At that 
time Lvov was part of Austria, but it is now in the Ukraine. His father was a captain in the Austrian army. 
Lukasiewicz became interested in mathematics while in high school. He studied mathematics and philosophy 
at the University of Lvov at both the undergraduate and graduate levels. After completing his doctoral work 
he became a lecturer there, and in 1911 he was appointed to a professorship. When the University of Warsaw 
was reopened as a Polish university in 1915, Lukasiewicz accepted an invitation to join the faculty. In 1919 
he served as the Polish Minister of Education. He returned to the position of professor at Warsaw University, 
where he remained from 1920 to 1939, serving as rector of the university twice. 

Lukasiewicz was one of the cofounders of the influential Warsaw School of Logic. He published his 
innovative text, Elements of Mathematical Logic, in 1928. With his influence, mathematical logic was made a 


required course for mathematics and science undergraduates in Poland. His lectures were considered excellent, even attracting 
students of the humanities. 

Lukasiewicz and his wife experienced great suffering during World War II, which he documented in a posthumously published 
autobiography. After the war they lived in exile in Belgium. Fortunately, in 1949 he was offered a position at the Royal Irish Academy 


in Dublin. 


Lukasiewicz worked on mathematical logic throughout his career. His work on a three-valued logic was an important contri- 
bution to the subject. Nevertheless, he is best known in the mathematical and computer science communities for his introduction of 
parenthesis-free notation, now called Polish notation. 


Extra 
Examples 


Exercises 


» 


11.3 Tree Traversal 819 


> 
“a 


DDE LN. 
i | ZN | 


i 
P q 


| p q 
q 
FIGURE 14 Constructing the rooted tree for a compound proposition. 
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Solution: The rooted tree for this compound proposition is constructed from the bottom up. 
First, subtrees for =p and ~q are formed (where ~ is considered a unary operator). Also, a 
subtree for p A q is formed. Then subtrees for =~(p A q) and (=p) V (^q) are constructed. Finally, 
these two subtrees are used to form the final rooted tree. The steps of this procedure are shown 
in Figure 14. 

The prefix, postfix, and infix forms of this expression are found by traversing this rooted 
tree in preorder, postorder, and inorder (including parentheses), respectively. These traversals 
give e 4 A pq V ap7q, pg \ ap7qa V <+, and (~(p A q)) e (p) V (-q)), respectively. < 


Because prefix and postfix expressions are unambiguous and because they can be evaluated 
easily without scanning back and forth, they are used extensively in computer science. Such 
expressions are especially useful in the construction of compilers. 


In Exercises 1-3 construct the universal address system for 3. 
the given ordered rooted tree. Then use this to order its ver- 
tices using the lexicographic order of their labels. 


1. 


4. Suppose that the address of the vertex v in the ordered 
rooted tree T is 3.4.5.2.4. 
a) At what level is v? 
b) What is the address of the parent of v? 
c) What is the least number of siblings v can have? 
d) What is the smallest possible number of vertices in T 

if v has this address? 

e) Find the other addresses that must occur. 

5. Suppose that the vertex with the largest address in an or- 
dered rooted tree T has address 2.3.4.3.1. Is it possible to 
determine the number of vertices in T? 
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Can the leaves of an ordered rooted tree have the follow- 
ing list of universal addresses? If so, construct such an 
ordered rooted tree. 
a LLLI 22L 212.213, 22, Sl, 
3 121;31:.22;32 
b) 2121.22, LA 22223232, 
2.4.2.1, 2.4.2.2, 3.1, 3.2.1, 3.2.2 
ce) 1.1, 1.2.1, 1.2.2, 1.2.2.1, 1.3, 1.4, 2, 3.1, 3.2, 4.1.1.1 


In Exercises 7-9 determine the order in which a preorder 
traversal visits the vertices of the given ordered rooted tree. 


7 


10. 


11. 


12. 


13. 


14. 


In which order are the vertices of the ordered rooted tree 
in Exercise 7 visited using an inorder traversal? 

In which order are the vertices of the ordered rooted tree 
in Exercise 8 visited using an inorder traversal? 

In which order are the vertices of the ordered rooted tree 
in Exercise 9 visited using an inorder traversal? 

In which order are the vertices of the ordered rooted tree 
in Exercise 7 visited using a postorder traversal? 

In which order are the vertices of the ordered rooted tree 
in Exercise 8 visited using a postorder traversal? 


15. 


16. 


17. 


18. 


19. 


* 20. 


*21. 


22. 


23. 


24. 


25. 


*26. 


In which order are the vertices of the ordered rooted tree 

in Exercise 9 visited using a postorder traversal? 

a) Represent the expression ((x+2) fT 3) * 
(y -—(3 +x)) — 5 using a binary tree. 

Write this expression in 

b) prefix notation. 

c) postfix notation. 

d) infix notation. 

a) Represent the expressions (x+xy)+(x/y) and 
x + ((xy + x)/y) using binary trees. 

Write these expressions in 

b) prefix notation. 

c) postfix notation. 

d) infix notation. 

a) Represent the compound propositions 7(p A q) © 
(=p V 7g) and (=p A (q © 7p)) V ~q using ordered 
rooted trees. 

Write these expressions in 

b) prefix notation. 

c) postfix notation. 

d) infix notation. 

a) Represent (A N B) — (A uU (B — A)) using an ordered 
rooted tree. 

Write this expression in 

b) prefix notation. 

c) postfix notation. 

d) infix notation. 

In how many ways can the string =p A q © “p V ~q be 

fully parenthesized to yield an infix expression? 

In how many ways can the string AN B—ANB-—A be 

fully parenthesized to yield an infix expression? 

Draw the ordered rooted tree corresponding to each of 

these arithmetic expressions written in prefix notation. 

Then write each expression using infix notation. 

a) +*+—-53214 

b) T+23-51 

c) */934+%24-76 

What is the value of each of these prefix expressions? 

a) —* 2/843 

b) T- *33%425 

c) +-132723/6-42 

d) x+3+313+333 

What is the value of each of these postfix expressions? 

a) 521--314++ 

b) 93/5+72-— x 

c) 32*2753-84/%«—- 

Construct the ordered rooted tree whose preorder traver- 

salis a, b, f, c, g, h, i, d, e, j, k, l, where a has four children, 

c has three children, j has two children, b and e have one 

child each, and all other vertices are leaves. 

Show that an ordered rooted tree is uniquely determined 

when a list of vertices generated by a preorder traversal 

of the tree and the number of children of each vertex are 
specified. 


* 27: 


28. 


29. 


Show that an ordered rooted tree is uniquely determined 
when a list of vertices generated by a postorder traversal 
of the tree and the number of children of each vertex are 
specified. 

Show that preorder traversals of the two ordered rooted 
trees displayed below produce the same list of vertices. 
Note that this does not contradict the statement in Exer- 
cise 26, because the numbers of children of internal ver- 
tices in the two ordered rooted trees differ. 


a a 


g h 
Show that postorder traversals of these two ordered 
rooted trees produce the same list of vertices. Note that 
this does not contradict the statement in Exercise 27, be- 
cause the numbers of children of internal vertices in the 


two ordered rooted trees differ. 


a a 
EO OX 
ĉ df g h c d bg h 
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Well-formed formulae in prefix notation over a set of sym- 
bols and a set of binary operators are defined recursively by 
these rules: 


30. 


*31, 


32. 


3 
3 


3. 


4. 


(i) if x is a symbol, then x is a well-formed formula in 
prefix notation; 

(ii) if X and Y are well-formed formulae and * is an op- 
erator, then * XY is a well-formed formula. 


Which of these are well-formed formulae over the sym- 
bols {x, y, z} and the set of binary operators {X, +, o}? 


a) X++xyx 
b) oxyXxz 
c) XoxzXXxy 


d) X+oxxoxxx 


Show that any well-formed formula in prefix notation 
over a set of symbols and a set of binary operators con- 
tains exactly one more symbol than the number of oper- 
ators. 


Give a definition of well-formed formulae in postfix no- 
tation over a set of symbols and a set of binary operators. 


Give six examples of well-formed formulae with three or 
more operators in postfix notation over the set of symbols 
{x, y, z} and the set of operators {+, x, o}. 


Extend the definition of well-formed formulae in prefix 
notation to sets of symbols and operators where the op- 
erators may not be binary. 


11.4.1 Introduction 


Consider the system of roads in Maine represented by the simple graph shown in Figure 1(a). 
The only way the roads can be kept open in the winter is by frequently plowing them. The 
highway department wants to plow the fewest roads so that there will always be cleared roads 
connecting any two towns. How can this be done? 

At least five roads must be plowed to ensure that there is a path between any two towns. 
Figure 1(b) shows one such set of roads. Note that the subgraph representing these roads is a 
tree, because it is connected and contains six vertices and five edges. 

This problem was solved with a connected subgraph with the minimum number of edges 
containing all vertices of the original simple graph. Such a graph must be a tree. 


Definition 1 


every vertex of G. 


Let G be a simple graph. A spanning tree of G is a subgraph of G that is a tree containing 
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Etna Old Town Etna Old Town 
a b e d 
Orono Orono 
Herman Bangor Herman; Bangor 
Hampden Hampden e g 
(a) (b) 
FIGURE 1 (a) A road system and (b) a set of FIGURE 2 The simple 
roads to plow. graph G. 


A simple graph with a spanning tree must be connected, because there is a path in the 
spanning tree between any two vertices. The converse is also true; that is, every connected 
simple graph has a spanning tree. We will give an example before proving this result. 


Find a spanning tree of the simple graph G shown in Figure 2. 


Solution: The graph G is connected, but it is not a tree because it contains simple circuits. 
Remove the edge {a, e}. This eliminates one simple circuit, and the resulting subgraph is still 
connected and still contains every vertex of G. Next remove the edge {e, f} to eliminate a second 
simple circuit. Finally, remove edge {c, g} to produce a simple graph with no simple circuits. 
This subgraph is a spanning tree, because it is a tree that contains every vertex of G. The se- 
quence of edge removals used to produce the spanning tree is illustrated in Figure 3. 


A ee 


Edge removed: {a, e} {e, f} {c, g} 


(a) (b) (c) 


FIGURE 3 Producing a spanning tree for G by removing edges that form simple 
circuits. 


c d 
e E 
e f 8g f 
a b c d 
e f 8 


FIGURE 4 Spanning trees of G. 


The tree shown in Figure 3 is not the only spanning tree of G. For instance, each of the trees 
shown in Figure 4 is a spanning tree of G. < 


A simple graph is connected if and only if it has a spanning tree. 
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Proof: First, suppose that a simple graph G has a spanning tree T. T contains every vertex of G. 
Furthermore, there is a path in T between any two of its vertices. Because T is a subgraph of G, 
there is a path in G between any two of its vertices. Hence, G is connected. 

Now suppose that G is connected. If G is not a tree, it must contain a simple circuit. Remove 
an edge from one of these simple circuits. The resulting subgraph has one fewer edge but still 
contains all the vertices of G and is connected. This subgraph is still connected because when 
two vertices are connected by a path containing the removed edge, they are connected by a path 
not containing this edge. We can construct such a path by inserting into the original path, at 
the point where the removed edge once was, the simple circuit with this edge removed. If this 
subgraph is not a tree, it has a simple circuit; so as before, remove an edge that is in a simple 
circuit. Repeat this process until no simple circuits remain. This is possible because there are 
only a finite number of edges in the graph. The process terminates when no simple circuits 
remain. A tree is produced because the graph stays connected as edges are removed. This tree 
is a Spanning tree because it contains every vertex of G. < 


Spanning trees are important in data networking, as Example 2 shows. 


IP Multicasting Spanning trees play an important role in multicasting over Internet Protocol 
(IP) networks. To send data from a source computer to multiple receiving computers, each of 
which is a subnetwork, data could be sent separately to each computer. This type of networking, 
called unicasting, is inefficient, because many copies of the same data are transmitted over the 
network. To make the transmission of data to multiple receiving computers more efficient, IP 
multicasting is used. With IP multicasting, a computer sends a single copy of data over the 
network, and as data reaches intermediate routers, the data are forwarded to one or more other 
routers so that ultimately all receiving computers in their various subnetworks receive these data. 
(Routers are computers that are dedicated to forwarding IP datagrams between subnetworks 
in a network. In multicasting, routers use Class D addresses, each representing a session that 
receiving computers may join; see Example 17 in Section 6.1.) 

For data to reach receiving computers as quickly as possible, there should be no loops 
(which in graph theory terminology are circuits or cycles) in the path that data take through 
the network. That is, once data have reached a particular router, data should never return to this 
router. To avoid loops, the multicast routers use network algorithms to construct a spanning tree 
in the graph that has the multicast source, the routers, and the subnetworks containing receiving 
computers as vertices, with edges representing the links between computers and/or routers. The 
root of this spanning tree is the multicast source. The subnetworks containing receiving com- 
puters are leaves of the tree. (Note that subnetworks not containing receiving stations are not 
included in the graph.) This is illustrated in Figure 5. < 


11.4.2 Depth-First Search 


The proof of Theorem 1 gives an algorithm for finding spanning trees by removing edges from 
simple circuits. This algorithm is inefficient, because it requires that simple circuits be identified. 
Instead of constructing spanning trees by removing edges, spanning trees can be built up by 
successively adding edges. Two algorithms based on this principle will be presented here. 

We can build a spanning tree for a connected simple graph using depth-first search. We will 
form a rooted tree, and the spanning tree will be the underlying undirected graph of this rooted 
tree. Arbitrarily choose a vertex of the graph as the root. Form a path starting at this vertex by 
successively adding vertices and edges, where each new edge is incident with the last vertex in 
the path and a vertex not already in the path. Continue adding vertices and edges to this path as 
long as possible. If the path goes through all vertices of the graph, the tree consisting of this path 
is a spanning tree. However, if the path does not go through all vertices, more vertices and edges 
must be added. Move back to the next to last vertex in the path, and, if possible, form a new path 
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FIGURE 5 A multicast spanning tree. 


starting at this vertex passing through vertices that were not already visited. If this cannot be 
done, move back another vertex in the path, that is, two vertices back in the path, and try again. 

Repeat this procedure, beginning at the last vertex visited, moving back up the path one 
vertex at a time, forming new paths that are as long as possible until no more edges can be 
added. Because the graph has a finite number of edges and is connected, this process ends with 
the production of a spanning tree. Each vertex that ends a path at a stage of the algorithm will 
be a leaf in the rooted tree, and each vertex where a path is constructed starting at this vertex 
will be an internal vertex. 

The reader should note the recursive nature of this procedure. Also, note that if the vertices 
in the graph are ordered, the choices of edges at each stage of the procedure are all determined 
when we always choose the first vertex in the ordering that is available. However, we will not 
always explicitly order the vertices of a graph. 

Depth-first search is also called backtracking, because the algorithm returns to vertices 
previously visited to add paths. Example 3 illustrates backtracking. 


Use depth-first search to find a spanning tree for the graph G shown in Figure 6. 


Solution: The steps used by depth-first search to produce a spanning tree of G are shown in 
Figure 7. We arbitrarily start with the vertex f. A path is built by successively adding edges 
incident with vertices not already in the path, as long as this is possible. This produces a path 
f. g, h, k, j (note that other paths could have been built). Next, backtrack to k. There is no 
path beginning at k containing vertices not already visited. So we backtrack to h. Form the 


5 (a) (b) (c) @) (©) 


FIGURE 6 The graphG. FIGURE7 Depth-first search of G. 
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path h, i. Then backtrack to h, and then to f. From f build the path f, d, e, c, a. Then backtrack 
to c and form the path c, b. This produces the spanning tree. < 


The edges selected by depth-first search of a graph are called tree edges. All other edges 
of the graph must connect a vertex to an ancestor or descendant of this vertex in the tree. These 
edges are called back edges. (Exercise 43 asks for a proof of this fact.) 


In Figure 8 we highlight the tree edges found by depth-first search starting at vertex f by showing 
them with heavy colored lines. The back edges (e, f) and (f, h) are shown with thinner black 
lines. < 


b g 


FIGURE 8 The tree edges and back edges 
of the depth-first search in example 4. 


We have explained how to find a spanning tree of a graph using depth-first search. However, 
our discussion so far has not brought out the recursive nature of depth-first search. To help 
make the recursive nature of the algorithm clear, we need a little terminology. We say that we 
explore from a vertex v when we carry out the steps of depth-first search beginning when v is 
added to the tree and ending when we have backtracked back to v for the last time. The key 
observation needed to understand the recursive nature of the algorithm is that when we add an 
edge connecting a vertex v to a vertex w, we finish exploring from w before we return to v to 
complete exploring from v. 

In Algorithm 1 we construct the spanning tree of a graph G with vertices v4, ..., v, by first 
selecting the vertex v, to be the root. We initially set T to be the tree with just this one vertex. 
At each step we add a new vertex to the tree T together with an edge from a vertex already 
in T to this new vertex and we explore from this new vertex. Note that at the completion of 
the algorithm, T contains no simple circuits because no edge is ever added that connects two 
vertices in the tree. Moreover, T remains connected as it is built. (These last two observations 
can be easily proved via mathematical induction.) Because G is connected, every vertex in G 
is visited by the algorithm and is added to the tree (as the reader should verify). It follows 
that T is a spanning tree of G. 


ALGORITHM 1 Depth-First Search. 


procedure DFS(G: connected graph with vertices v4, V3, ..., V,) 
T := tree consisting only of the vertex v, 
visit(V,) 


procedure visit(v: vertex of G) 

for each vertex w adjacent to v and not yet in T 
add vertex w and edge {v, w} to T 
visit(w) 
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We now analyze the computational complexity of the depth-first search algorithm. The key 
observation is that for each vertex v, the procedure visit(v) is called when the vertex v is first 
encountered in the search and it is not called again. Assuming that the adjacency lists for G are 
available (see Section 10.3), no computations are required to find the vertices adjacent to v. As 
we follow the steps of the algorithm, we examine each edge at most twice to determine whether 
to add this edge and one of its endpoints to the tree. Consequently, the procedure DFS constructs 
a spanning tree using O(e), or O(n’), steps where e and n are the number of edges and vertices 
in G, respectively. [Note that a step involves examining a vertex to see whether it is already in 
the spanning tree as it is being built and adding this vertex and the corresponding edge if the 
vertex is not already in the tree. We have also made use of the inequality e < n(n — 1)/2, which 
holds for any simple graph.] 

Depth-first search can be used as the basis for algorithms that solve many different problems. 
For example, it can be used to find paths and circuits in a graph, it can be used to determine 
the connected components of a graph, and it can be used to find the cut vertices of a connected 
graph. As we will see, depth-first search is the basis of backtracking techniques used to search 
for solutions of computationally difficult problems. (See [GrYe05], [Ma89], and [CoLeRiSt09] 
for a discussion of algorithms based on depth-first search.) 


11.4.3 Breadth-First Search 


We can also produce a spanning tree of a simple graph by the use of breadth-first search. 
Again, a rooted tree will be constructed, and the underlying undirected graph of this rooted tree 
forms the spanning tree. Arbitrarily choose a root from the vertices of the graph. Then add all 
edges incident to this vertex. The new vertices added at this stage become the vertices at level 1 
in the spanning tree. Arbitrarily order them. Next, for each vertex at level 1, visited in order, 
add each edge incident to this vertex to the tree as long as it does not produce a simple circuit. 
Arbitrarily order the children of each vertex at level 1. This produces the vertices at level 2 
in the tree. Follow the same procedure until all the vertices in the tree have been added. The 
procedure ends because there are only a finite number of edges in the graph. A spanning tree is 
produced because we have produced a tree containing every vertex of the graph. An example 
of breadth-first search is given in Example 5. 


Use breadth-first search to find a spanning tree for the graph shown in Figure 9. 


Solution: The steps of the breadth-first search procedure are shown in Figure 10. We choose the 
vertex e to be the root. Then we add edges incident with all vertices adjacent to e, so edges from 
e to b, d, f, andi are added. These four vertices are at level 1 in the tree. Next, add the edges 
from these vertices at level 1 to adjacent vertices not already in the tree. Hence, the edges from 
b to a and c are added, as are edges from d to h, from f to j and g, and from i to k. The new 
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FIGURE 10 Breadth-first search of G. 


vertices a, c, h, j, g, and k are at level 2. Next, add edges from these vertices to adjacent vertices 
not already in the graph. This adds edges from g to / and from k to m. < 


We describe breadth-first search in pseudocode as Algorithm 2. In this algorithm, we as- 
sume the vertices of the connected graph G are ordered as v}, v2, ..., v,. In the algorithm we 
use the term “process” to describe the procedure of adding new vertices, and corresponding 
edges, to the tree adjacent to the current vertex being processed as long as a simple circuit is 
not produced. 


ALGORITHM 2 Breadth-First Search. 


procedure BFS(G: connected graph with vertices v}, vz, ...,V,,) 
T := tree consisting only of vertex v, 
L := empty list 
put v; in the list L of unprocessed vertices 
while L is not empty 
remove the first vertex, v, from L 
for each neighbor w of v 
if w is not in L and not in T then 
add w to the end of the list L 
add w and edge {v, w} to T 


We now analyze the computational complexity of breadth-first search. For each vertex v in 
the graph we examine all vertices adjacent to v and we add each vertex not yet visited to the 
tree T. Assuming we have the adjacency lists for the graph available, no computation is required 
to determine which vertices are adjacent to a given vertex. As in the analysis of the depth-first 
search algorithm, we see that we examine each edge at most twice to determine whether we 
should add this edge and its endpoint not already in the tree. It follows that the breadth-first 
search algorithm uses O(e) or O(n?) steps. 

Breadth-first search is one of the most useful algorithms in graph theory. In particular, it can 
serve as the basis for algorithms that solve a wide variety of problems. For example, algorithms 
that find the connected components of a graph, that determine whether a graph is bipartite, and 
that find the path with the fewest edges between two vertices in a graph can all be built using 
breadth-first search. 


Breadth-First Search versus Depth-First Search We have introduced two widely used al- 
gorithms for constructing spanning trees of graphs, breadth-first search (BFS) and depth-first 
search (DFS). Both algorithms can be used to produce a spanning tree when given a connected 
graph. But why might it be better to use one rather than the other? 


828 


11/ Trees 


EXAMPLE 6 


Although both BFS and DFS can be used to solve the same problems, there are theoretical 
and practical reasons for choosing between them. To solve a particular type of problem one of 
these search algorithms can be easier to apply, or can provide more insight, than the other. For 
some problems, BFS can be easier to use than DFS because BFS partitions the vertices of a 
graph into levels, which tell us how far away vertices are from the root. Also, edges connect 
vertices at the same level or at adjoining levels (see Exercise 34). On the other hand, there are 
many types of problems that are more naturally solved using DFS, such as those discussed in 
Examples 6-8. Generally, DFS is a better choice when we need to explore a graph by exploring 
deeper rather than by exploring the graph systematically level by level. The structure obtained 
when using DFS (see Exercise 51) can also be used when solving problems. 

Both BFS and DFS are extensively used in practice. The choice for which to use often 
depends on implementation details, such as the data structures used. Time and space consider- 
ations are paramount, especially when the problems being solved involved huge graphs. Also, 
keep in mind that we often solve problems using graph searching without completing the task 
of finding a spanning tree. When we use BFS on a graph that is dense, we spend a lot of time 
and use a lot of space to explore the graph level by level. In such situations, it can be preferable 
to use DFS to quickly reach vertices far away from the root. For a sparse graph, however, it may 
be more efficient to explore a graph level by level. 


11.4.4 Backtracking Applications 


There are problems that can be solved only by performing an exhaustive search of all possible 
solutions. One way to search systematically for a solution is to use a decision tree, where each 
internal vertex represents a decision and each leaf a possible solution. To find a solution via 
backtracking, first make a sequence of decisions in an attempt to reach a solution as long as this 
is possible. The sequence of decisions can be represented by a path in the decision tree. Once 
it is known that no solution can result from any further sequence of decisions, backtrack to the 
parent of the current vertex and work toward a solution with another series of decisions, if this 
is possible. The procedure continues until a solution is found, or it is established that no solution 
exists. Examples 6 to 8 illustrate the usefulness of backtracking. 


Graph Colorings How can backtracking be used to decide whether a graph can be colored 
using n colors? 


Solution: We can solve this problem using backtracking in the following way. First pick some 
vertex a and assign it color 1. Then pick a second vertex b, and if b is not adjacent to a, assign it 
color 1. Otherwise, assign color 2 to b. Then go on to a third vertex c. Use color 1, if possible, 
for c. Otherwise use color 2, if this is possible. Only if neither color 1 nor color 2 can be used 
should color 3 be used. Continue this process as long as it is possible to assign one of the n 
colors to each additional vertex, always using the first allowable color in the list. If a vertex is 
reached that cannot be colored by any of the n colors, backtrack to the last assignment made and 
change the coloring of the last vertex colored, if possible, using the next allowable color in the 
list. If it is not possible to change this coloring, backtrack farther to previous assignments, one 
step back at a time, until it is possible to change a coloring of a vertex. Then continue assigning 
colors of additional vertices as long as possible. If a coloring using n colors exists, backtracking 
will produce it. (Unfortunately this procedure can be extremely inefficient.) 

In particular, consider the problem of coloring the graph shown in Figure 11 with three 
colors. The tree shown in Figure 11 illustrates how backtracking can be used to construct a 3- 
coloring. In this procedure, red is used first, then blue, and finally green. This simple example 
can obviously be done without backtracking, but it is a good illustration of the technique. 

In this tree, the initial path from the root, which represents the assignment of red to a, leads 
to a coloring with a red, b blue, c red, and d green. It is impossible to color e using any of 
the three colors when a, b, c, and d are colored in this way. So, backtrack to the parent of the 
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FIGURE 11 Coloring a graph using backtracking. 


vertex representing this coloring. Because no other color can be used for d, backtrack one more 
level. Then change the color of c to green. We obtain a coloring of the graph by then assigning 
red to d and green to e. < 


The n-Queens Problem The n-queens problem asks how n queens can be placed on ann X n 
chessboard so that no two queens can attack one another. How can backtracking be used to solve 
the n-queens problem? 


Solution: To solve this problem we must find n positions on an n X n chessboard so that no 
two of these positions are in the same row, same column, or in the same diagonal [a diagonal 
consists of all positions (i, j) with i +j = m for some m, or i — j = m for some m]. We will use 
backtracking to solve the n-queens problem. We start with an empty chessboard. At stage k + 1 
we attempt putting an additional queen on the board in the (k + 1)st column, where there are 
already queens in the first k columns. We examine squares in the (k + 1)st column starting with 
the square in the first row, looking for a position to place this queen so that it is not in the same 
row or on the same diagonal as a queen already on the board. (We already know it is not in the 
same column.) If it is impossible to find a position to place the queen in the (k + 1)st column, 
backtrack to the placement of the queen in the kth column, and place this queen in the next 
allowable row in this column, if such a row exists. If no such row exists, backtrack further. 

In particular, Figure 12 displays a backtracking solution to the four-queens problem. In this 
solution, we place a queen in the first row and column. Then we put a queen in the third row of 
the second column. However, this makes it impossible to place a queen in the third column. So 
we backtrack and put a queen in the fourth row of the second column. When we do this, we can 
place a queen in the second row of the third column. But there is no way to add a queen to the 
fourth column. This shows that no solution results when a queen is placed in the first row and 
column. We backtrack to the empty chessboard, and place a queen in the second row of the first 
column. This leads to a solution as shown in Figure 12. < 


Sums of Subsets Consider this problem. Given a set of positive integers x4, Xz, ...,x,, find a 
subset of this set of integers that has M as its sum. How can backtracking be used to solve this 
problem? 


Solution: We start with a sum with no terms. We build up the sum by successively adding terms. 
An integer in the sequence is included if the sum remains less than M when this integer is added 
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FIGURE 12 A backtracking solution of the four-queens problem. 


Sum = 0 


{31, 7} {31, 5} {27, 11} {27, 7} 
Sum = 38 Sum = 36 Sum = 38 Sum = 34 


{27, 7, 5} 
Sum = 39 


FIGURE 13 Find a sum equal to 39 using backtracking. 


to the sum. If a sum is reached such that the addition of any term is greater than M, backtrack 
by dropping the last term of the sum. 

Figure 13 displays a backtracking solution to the problem of finding a subset of 
{31, 27, 15, 11, 7,5} with the sum equal to 39. < 


11.4.5  Depth-First Search in Directed Graphs 


We can easily modify both depth-first search and breadth-first search so that they can run given 
a directed graph as input. However, the output will not necessarily be a spanning tree, but rather 
a spanning forest. In both algorithms we can add an edge only when it is directed away from 
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FIGURE 14 _ Depth-first search of a directed graph. 


the vertex that is being visited and to a vertex not yet added. If at a stage of either algorithm we 
find that no edge exists starting at a vertex already added to one not yet added, the next vertex 
added by the algorithm becomes the root of a new tree in the spanning forest. This is illustrated 
in Example 9. 


What is the output of depth-first search given the graph G shown in Figure 14(a) as input? 


Solution: We begin the depth-first search at vertex a and add vertices b, c, and g and the cor- 
responding edges where we are blocked. We backtrack to c but we are still blocked, and then 
backtrack to b, where we add vertices f and e and the corresponding edges. Backtracking takes 
us all the way back to a. We then start a new tree at d and add vertices h, l, k, and j and the 
corresponding edges. We backtrack to k, then /, then h, and back to d. Finally, we start a new 
tree at i, completing the depth-first search. The output is shown in Figure 14(b). < 


Depth-first search in directed graphs is the basis of many algorithms (see [GrYe05], [Ma89], 
and [CoLeRiSt09]). It can be used to determine whether a directed graph has a circuit, it can 
be used to carry out a topological sort of a graph, and it can also be used to find the strongly 
connected components of a directed graph. 

We conclude this section with an application of depth-first search and breadth-first search 
to search engines on the web. 


Web Crawlers To index websites, search engines such as Google and Bing systematically 
explore the web starting at known sites. These search engines use programs called web crawlers 
(or spiders or bots) to visit websites and analyze their contents. Web crawlers use both depth- 
first searching and breadth-first searching to create indices. As described in Example 5 in Sec- 
tion 10.1, web pages and links between them can be modeled by a directed graph called the 
web graph. Web pages are represented by vertices and links are represented by directed edges. 
When DFS is used to explore the web, an initial web page is selected, a link is followed to a 
second web page (if there is such a link), a link on the second web page is followed to a third 
web page, if there is such a link, and so on, until a page with no new links is found. Backtracking 
is then used to examine links at the previous level to look for new links, and so on. (Because 
of practical limitations, web crawlers have limits to the depth they search in depth-first search.) 
Using BFS an initial web page is selected and a link on this page is followed to a second web 
page, then a second link on the initial page is followed (if it exists), and so on, until all links of 
the initial page have been followed. Then links on the pages one level down are followed, page 
by page, and so on. 

Search engines have developed sophisticated strategies for web crawlers that take advantage 
of BFS and DFS. (Google’s web crawler is called Googlebot.) They use BFS, starting on high 
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quality web pages, called seeds, and explore all links on these pages. As the web crawler pro- 
ceeds, it adds the urls of all links on the new pages it visits, adding them to the crawl frontier. 
These urls are ordered according to a particular policy, determining the order in which websites 


are explored. 


When seeds have been selected well, Google’s approach has been found to quickly reach 
high quality pages that have many links pointing to them. However, the quality of pages reached 
decreases as the web crawl continues. This reaches popular web pages well, but it does not reach 
many other useful, but less popular, web pages. If seed pages do not yield good results, DFS 
can be used to find candidates for high quality pages, starting at the seeds or other web pages. 
Also, DFS can be used to reach parts of the web not reached by BFS when it is restricted to a 


particular number of levels. 


Exercises 


1. How many edges must be removed from a connected 
graph with n vertices and m edges to produce a spanning 
tree? 

In Exercises 2—6 find a spanning tree for the graph shown by 
removing edges in simple circuits. 


2. 4 b č 
d e 
3. 
a 
d 
&§ 
4. 


< 


7. Find a spanning tree for each of these graphs. 
a) K; b) Ky4 c) Kis 
d) Q; e) Cs f) W; 
In Exercises 8—10 draw all the spanning trees of the given 
simple graphs. 
8. a 


b ç 
9. a b f g 
c d e h 


10. 


*11. How many different spanning trees does each of these 
simple graphs have? 
a) K; b) K, c) Ky 5 d) C; 

*12. How many nonisomorphic spanning trees does each of 
these simple graphs have? 
a) K; b) K, c) K; 


In Exercises 13—15 use depth-first search to produce a span- 
ning tree for the given simple graph. Choose a as the root of 
this spanning tree and assume that the vertices are ordered 
alphabetically. 


13. a e h i 


14. 


15. 


a 


16. Use breadth-first search to produce a spanning tree for 
each of the simple graphs in Exercises 13—15. Choose a 
as the root of each spanning tree. 


17. Use depth-first search to find a spanning tree of each of 
these graphs. 
a) W; (see Example 7 of Section 10.2), starting at the 
vertex of degree 6 


b) K; 
c) K; 4, starting at a vertex of degree 3 


d) Q; 
18. Use breadth-first search to find a spanning tree of each of 
the graphs in Exercise 17. 


19. Describe the trees produced by breadth-first search and 
depth-first search of the wheel graph W,, starting at the 


vertex of degree n, where n is an integer with n > 3. (See 
Example 7 of Section 10.2.) Justify your answers. 


20. Describe the trees produced by breadth-first search and 
depth-first search of the complete graph K,,, where n is a 
positive integer. Justify your answers. 


21. Describe the trees produced by breadth-first search and 
depth-first search of the complete bipartite graph Kn 
starting at a vertex of degree m, where m and n are posi- 
tive integers. Justify your answers. 
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22. Describe the tree produced by breadth-first search and 
depth-first search for the n-cube graph Q,„, where n is a 
positive integer. 


23. Suppose that an airline must reduce its flight schedule to 
save money. If its original routes are as illustrated here, 
which flights can be discontinued to retain service be- 
tween all pairs of cities (where it may be necessary to 
combine flights to fly from one city to another)? 


24. Explain how breadth-first search or depth-first search can 
be used to order the vertices of a connected graph. 


*25. Show that the length of the shortest path between ver- 
tices v and u in a connected simple graph equals the level 
number of u in the breadth-first spanning tree of G with 
root v. 


26. Use backtracking to try to find a coloring of each of the 
graphs in Exercises 7-9 of Section 10.8 using three col- 
ors. 


27. Use backtracking to solve the n-queens problem for these 
values of n. 
a) n=3 b) n=5 c) n=6 

28. Use backtracking to find a subset, if it exists, of the set 
{27, 24, 19, 14, 11, 8} with sum 
a) 20. b) 41. c) 60. 

29. Explain how backtracking can be used to find a Hamilton 
path or circuit in a graph. 


30. a) Explain how backtracking can be used to find the way 
out of a maze, given a starting position and the exit 
position. Consider the maze divided into positions, 
where at each position the set of available moves in- 
cludes one to four possibilities (up, down, right, left). 


b) Find a path from the starting position marked by X to 
the exit in this maze. 


X 


Exit 
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A spanning forest of a graph G is a forest that contains every 
vertex of G such that two vertices are in the same tree of the 
forest when there is a path in G between these two vertices. 


31. 
32. 
33. 


34. 


35. 


36. 


37. 


38. 


39. 


40. 


41. 


42. 


43. 


44. 


45. 


46. 


47. 


48. 


49. 


Show that every finite simple graph has a spanning forest. 
How many trees are in the spanning forest of a graph? 
How many edges must be removed to produce the span- 
ning forest of a graph with n vertices, m edges, and c con- 
nected components? 

Let G be a connected graph. Show that if T is a spanning 
tree of G constructed using breadth-first search, then an 
edge of G not in T must connect vertices at the same level 
or at levels that differ by 1 in this spanning tree. 

Explain how to use breadth-first search to find the length 
of a shortest path between two vertices in an undirected 
graph. 

Devise an algorithm based on breadth-first search that de- 
termines whether a graph has a simple circuit, and if so, 
finds one. 

Devise an algorithm based on breadth-first search for 
finding the connected components of a graph. 

Explain how breadth-first search and how depth-first 
search can be used to determine whether a graph is bi- 
partite. 

Which connected simple graphs have exactly one span- 
ning tree? 

Devise an algorithm for constructing the spanning forest 
of a graph based on deleting edges that form simple cir- 
cuits. 

Devise an algorithm for constructing the spanning forest 
of a graph based on depth-first searching. 

Devise an algorithm for constructing the spanning forest 
of a graph based on breadth-first searching. 

Let G be a connected graph. Show that if T is a span- 
ning tree of G constructed using depth-first search, then 
an edge of G not in T must be a back edge, that is, it must 
connect a vertex to one of its ancestors or one of its de- 
scendants in T. 

When must an edge of a connected simple graph be in 
every spanning tree for this graph? 

For which graphs do depth-first search and breadth-first 
search produce identical spanning trees no matter which 
vertex is selected as the root of the tree? Justify your an- 
swer. 

Use Exercise 43 to prove that if G is a connected, simple 
graph with n vertices and G does not contain a simple 
path of length k then it contains at most (k — 1)n edges. 
Use mathematical induction to prove that breadth-first 
earch visits vertices in order of their level in the resulting 
spanning tree. 

Use pseudocode to describe a variation of depth-first 
search that assigns the integer n to the nth vertex visited in 
the search. Show that this numbering corresponds to the 
numbering of the vertices created by a preorder traversal 
of the spanning tree. 

Use pseudocode to describe a variation of breadth-first 
search that assigns the integer m to the mth vertex visited 
in the search. 


* 50. 


51. 


* 52. 


Suppose that G is a directed graph and T is a spanning 
tree constructed using breadth-first search. Show that ev- 
ery edge of G has endpoints that are at the same level or 
one level higher or lower. 


Show that if G is a directed graph and T is a spanning 
tree constructed using depth-first search, then every edge 
not in the spanning tree is a forward edge connecting 
an ancestor to a descendant, a back edge connecting a 
descendant to an ancestor, or a cross edge connecting a 
vertex to a vertex in a previously visited subtree. 


Describe a variation of depth-first search that assigns the 
smallest available positive integer to a vertex when the 
algorithm is totally finished with this vertex. Show that 
in this numbering, each vertex has a larger number than 
its children and that the children have increasing numbers 
from left to right. 


Let T, and T, be spanning trees of a graph. The distance be- 
tween T, and T, is the number of edges in T) and T, that are 
not common to T} and T). 


53. 


* 54. 


TESS, 


* 56. 


Find the distance between each pair of spanning trees 
shown in Figures 3(c) and 4 of the graph G shown in Fig- 
ure 2. 


Suppose that T,, T), and T are spanning trees of the sim- 
ple graph G. Show that the distance between T, and T3 
does not exceed the sum of the distance between T, and 
T, and the distance between T, and T}. 


Suppose that T) and T, are spanning trees of a simple 
graph G. Moreover, suppose that e; is an edge in T, that 
is not in 7,. Show that there is an edge e, in T, that is not 
in T} such that T} remains a spanning tree if e, is removed 
from it and e, is added to it, and T, remains a spanning 
tree if e, is removed from it and e, is added to it. 


Show that it is possible to find a sequence of spanning 
trees leading from any spanning tree to any other by suc- 
cessively removing one edge and adding another. 


A rooted spanning tree of a directed graph is a rooted tree 
containing edges of the graph such that every vertex of the 
graph is an endpoint of one of the edges in the tree. 


57. 


* 58. 


*59, 


*60. 


*61. 


For each of the directed graphs in Exercises 18-23 of 
Section 10.5 either find a rooted spanning tree of the 
graph or determine that no such tree exists. 


Show that a connected directed graph in which each ver- 
tex has the same in-degree and out-degree has a rooted 
spanning tree. [Hint: Use an Euler circuit. ] 


Give an algorithm to build a rooted spanning tree for con- 
nected directed graphs in which each vertex has the same 
in-degree and out-degree. 


Show that if G is a directed graph and T is a spanning 
tree constructed using depth-first search, then G contains 
a circuit if and only if G contains a back edge (see Exer- 
cise 51) relative to the spanning tree T. 


Use Exercise 60 to construct an algorithm for determin- 
ing whether a directed graph contains a circuit. 
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11.5.1 Introduction 


A company plans to build a communications network connecting its five computer centers. Any 
pair of these centers can be linked with a leased telephone line. Which links should be made to 
ensure that there is a path between any two computer centers so that the total cost of the network 
is minimized? We can model this problem using the weighted graph shown in Figure 1, where 
vertices represent computer centers, edges represent possible leased lines, and the weights on 
edges are the monthly lease rates of the lines represented by the edges. We can solve this problem 
by finding a spanning tree so that the sum of the weights of the edges of the tree is minimized. 
Such a spanning tree is called a minimum spanning tree. 


$2000 


New York 


Chicago 


San Francisco 


FIGURE 1 A weighted graph showing monthly lease 
costs for lines in a computer network. 


11.5.2 Algorithms for Minimum Spanning Trees 


A wide variety of problems are solved by finding a spanning tree in a weighted graph such that 
the sum of the weights of the edges in the tree is a minimum. 


A minimum spanning tree in a connected weighted graph is a spanning tree that has the 
smallest possible sum of weights of its edges. 


We will present two algorithms for constructing minimum spanning trees. Both proceed 
by successively adding edges of smallest weight from those edges with a specified property 
that have not already been used. Both are greedy algorithms. Recall from Section 3.1 that a 
greedy algorithm is a procedure that makes an optimal choice at each of its steps. Optimizing 
at each step does not guarantee that the optimal overall solution is produced. However, the 
two algorithms presented in this section for constructing minimum spanning trees are greedy 
algorithms that do produce optimal solutions. 

The first algorithm that we will discuss was originally discovered by the Czech mathe- 
matician Vojtéch Jarnik in 1930, who described it in a paper in an obscure Czech journal. The 
algorithm became well known when it was rediscovered in 1957 by Robert Prim. Because of 
this, it is known as Prim’s algorithm (and sometimes as the Prim-Jarnik algorithm). Begin 
by choosing any edge with smallest weight, putting it into the spanning tree. Successively add to 
the tree edges of minimum weight that are incident to a vertex already in the tree, never forming 
a simple circuit with those edges already in the tree. Stop when n — 1 edges have been added. 

Later in this section, we will prove that this algorithm produces a minimum spanning tree for 
any connected weighted graph. Algorithm 1 gives a pseudocode description of Prim’s algorithm. 
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EXAMPLE 1 


EXAMPLE 2 


San Francisco 


$9, 
20 Denver 


ALGORITHM 1 Prim’s Algorithm. 


procedure Prim(G: weighted connected undirected graph with n vertices) 
T := a minimum-weight edge 
for i:= 1 ton- 2 
e := an edge of minimum weight incident to a vertex in T and not forming a 
simple circuit in T if added to T 
T := T with e added 
return T {T is a minimum spanning tree of G} 


Note that the choice of an edge to add at a stage of the algorithm is not determined when there 
is more than one edge with the same weight that satisfies the appropriate criteria. We need to 
order the edges to make the choices deterministic. We will not worry about this in the remainder 
of the section. Also note that there may be more than one minimum spanning tree for a given 
connected weighted simple graph. (See Exercise 9.) Examples 1 and 2 illustrate how Prim’s 
algorithm is used. 


Use Prim’s algorithm to design a minimum-cost communications network connecting all the 
computers represented by the graph in Figure 1. 


Solution: We solve this problem by finding a minimum spanning tree in the graph in Figure 1. 
Prim’s algorithm is carried out by choosing an initial edge of minimum weight and successively 
adding edges of minimum weight that are incident to a vertex in the tree and that do not form 
simple circuits. The edges in color in Figure 2 show a minimum spanning tree produced by 
Prim’s algorithm, with the choice made at each step displayed. < 


Use Prim’s algorithm to find a minimum spanning tree in the graph shown in Figure 3. 


Solution: A minimum spanning tree constructed using Prim’s algorithm is shown in Figure 4. 
The successive edges chosen are displayed. < 


$1200 


Choice Edge Cost 
1 {Chicago, Atlanta} $ 700 
2 {Atlanta, New York} $ 800 
3 {Chicago, San Francisco} $1200 
4 {San Francisco, Denver} $ 900 
Total: $3600 i j k I 
FIGURE 2 A minimum spanning tree for the weighted FIGURE 3 A weighted graph. 


graph in Figure 1. 


Links » 


Courtesy of Robert Clay 
Prim 


Courtesy of Joseph Kruskal 
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Choice Edge Weight 

a 2 b 3 c 1 d 1 {b, f} 1 
° ii hd 2 {a, b} 2 
3 {hi} 2 

3 1 2 5 4 {a, e} 3 
5 {i, j} 3 

ee 2 4 ; a 3 eh 9 U 8] : 
7 {c, g} 2 

8 {c, d} 1 

4 2 4 3 9 {g, h} 3 
10 {h, 1} 3 
=e a | 11 {k, 1} Ea 
t J k L Total: 24 


(a) (b) 


FIGURE 4 A minimum spanning tree produced using Prim’s algorithm. 


The second algorithm we will discuss was discovered by Joseph Kruskal in 1956, although 
the basic ideas it uses were described much earlier. To carry out Kruskal’s algorithm, choose 
an edge in the graph with minimum weight. 

Successively add edges with minimum weight that do not form a simple circuit with those 
edges already chosen. Stop after n — 1 edges have been selected. 

The proof that Kruskal’s algorithm produces a minimum spanning tree for every connected 
weighted graph is left as an exercise. Pseudocode for Kruskal’s algorithm is given in Algo- 
rithm 2. 


ALGORITHM 2 Kruskal’s Algorithm. 


procedure Kruskal(G: weighted connected undirected graph with n vertices) 
T := empty graph 
fori := 1ton-1 
e := any edge in G with smallest weight that does not form a simple circuit 
when added to T 
T := T with e added 
return T {T is a minimum spanning tree of G} 


ROBERT CLAY PRIM (BORN 1921) Robert Prim, born in Sweetwater, Texas, received his B.S. in electrical 
engineering in 1941 and his Ph.D. in mathematics from Princeton University in 1949. He was an engineer at 
the General Electric Company from 1941 until 1944, an engineer and mathematician at the United States Naval 
Ordnance Lab from 1944 until 1949, and a research associate at Princeton University from 1948 until 1949. 
Among the other positions he has held are director of mathematics and mechanics research at Bell Telephone 
Laboratories from 1958 until 1961 and vice president of research at Sandia Corporation. He is currently retired. 


JOSEPH BERNARD KRUSKAL (1928-2010) Joseph Kruskal was born in New York City, where his father 
was a fur dealer and his mother promoted the art of origami on early television. Kruskal attended the Univer- 
sity of Chicago and received his Ph.D. from Princeton University in 1954. He was an instructor in mathemat- 
ics at Princeton and at the University of Wisconsin, and later he was an assistant professor at the University 
of Michigan. In 1959 he became a member of the technical staff at Bell Laboratories, where he worked un- 
til his retirement in the late 1990s. Kruskal discovered his algorithm for producing minimum spanning trees 


when he was a second-year graduate student. He was not sure his 24-page paper on this subject was worthy 
of publication, but was convinced by others to submit it. His research interests included statistical linguistics 
and psychometrics. Besides his work on minimum spanning trees, Kruskal is also known for contributions to 
multidimensional scaling. It is noteworthy that Joseph Kruskal’s two brothers, Martin and William, also were 
well known mathematicians. 
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EXAMPLE 3 


Extra > 
Examples 


Links » 


Choice Edge Weight 

a 2 b 3 G 1 d 1 {c, d} 1 
e e ° r 2 {k, 1} 1 
3 {b, f} 1 

3 1 2 5 4 {c, g} 2 
5 {a, b} 2 

ce 2 pi 2 2 3 eh ç J) 2 
7 {b, c} 3 

8 {j, k} 3 

4 2 4 3 9 {g, h} 3 
10 {i,j} 3 
e 3 e è l e 11 {a, e} EA 
i j k l Total: 24 


(a) (b) 


FIGURE 5 A minimum spanning tree produced by Kruskal’s algorithm. 


The reader should note the difference between Prim’s and Kruskal’s algorithms. In Prim’s 
algorithm edges of minimum weight that are incident to a vertex already in the tree, and not 
forming a circuit, are chosen; whereas in Kruskal’s algorithm edges of minimum weight that 
are not necessarily incident to a vertex already in the tree, and that do not form a circuit, are 
chosen. Note that as in Prim’s algorithm, if the edges are not ordered, there may be more than 
one choice for the edge to add at a stage of this procedure. Consequently, the edges need to be 
ordered for the procedure to be deterministic. Example 3 illustrates how Kruskal’s algorithm is 
used. 


Use Kruskal’s algorithm to find a minimum spanning tree in the weighted graph shown in 
Figure 3. 


Solution: A minimum spanning tree and the choices of edges at each stage of Kruskal’s algo- 
rithm are shown in Figure 5. < 


We will now prove that Prim’s algorithm produces a minimum spanning tree of a connected 
weighted graph. 


Proof: Let G be a connected weighted graph. Suppose that the successive edges chosen by 
Prim’s algorithm are e}, ez, ..., €,_,. Let S be the tree with e}, e2, ..., €„—1 as its edges, and let S, 
be the tree with e}, e2, ..., e, as its edges. Let T be a minimum spanning tree of G containing the 
edges €], €z, ..., €p, where k is the maximum integer with the property that a minimum spanning 
tree exists containing the first k edges chosen by Prim’s algorithm. The theorem follows if we 
can show that S = T. 

Suppose that S #7, so that k<n—1. Consequently, T contains e}, €>, ..., €p, but 
not e,,,. Consider the graph made up of T together with e,,,. Because this graph is connected 
and has n edges, too many edges to be a tree, it must contain a simple circuit. This simple cir- 
cuit must contain e,,, because there was no simple circuit in T. Furthermore, there must be an 


HISTORICAL NOTE Joseph Kruskal and Robert Prim developed their algorithms for constructing minimum 
spanning trees in the mid-1950s. However, they were not the first people to discover such algorithms. For 
example, the work of the anthropologist Jan Czekanowski, in 1909, contains many of the ideas required to find 
minimum spanning trees. In 1926, Otakar Boruvka described methods for constructing minimum spanning trees 
in work relating to the construction of electric power networks, and as mentioned in the text what is now called 
Prim’s algorithm was discovered by Vojtéch Jarnik in 1930. 
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edge in the simple circuit that does not belong to S,,, because S;,,, is a tree. By starting at an 
endpoint of e,,, that is also an endpoint of one of the edges e}, ..., e,, and following the circuit 
until it reaches an edge not in S,,,, we can find an edge e not in S,,, that has an endpoint that 
is also an endpoint of one of the edges e}, ez, ..., €z. 

By deleting e from T and adding e,,,, we obtain a tree T” with n — 1 edges (it is a tree 
because it has no simple circuits). Note that the tree T’ contains e}, ez, ..., €% €,,,. Furthermore, 
because e,,, was chosen by Prim’s algorithm at the (k + 1)st step, and e was also available at 
that step, the weight of e,,, is less than or equal to the weight of e. From this observation, it 
follows that T’ is also a minimum spanning tree, because the sum of the weights of its edges 
does not exceed the sum of the weights of the edges of T. This contradicts the choice of k as 
the maximum integer such that a minimum spanning tree exists containing e€}, ..., e,. Hence, 
k =n- l, and S = T. It follows that Prim’s algorithm produces a minimum spanning tree. < 


It can be shown (see [CoLeRiSt09]) that to find a minimum spanning tree of a graph with 
m edges and n vertices, Kruskal’s algorithm can be carried out using O(m log m) operations and 
Prim’s algorithm can be carried out using O(m log n) operations. Consequently, it is preferable 
to use Kruskal’s algorithm for graphs that are sparse, that is, where m is very small compared to 
C(n, 2) = n(n — 1)/2, the total number of possible edges in an undirected graph with n vertices. 
Otherwise, there is little difference in the complexity of these two algorithms. 


1. The roads represented by this graph are all unpaved. The 


3. a 5 b 4 c 
lengths of the roads between pairs of towns are repre- 
sented by edge weights. Which roads should be paved | o/s \¢ 
so that there is a path of paved roads between each F 


pair of towns so that a minimum road length is paved? 


f 
(Note: These towns are in Nevada.) 6 RA 4 
4 h 2 i 


& 


Manhattan 


Dyer 


Beatty 


In Exercises 2—4 use Prim’s algorithm to find a minimum 
spanning tree for the given weighted graph. 5. Use Kruskal’s algorithm to design the communications 


network described at the beginning of the section. 


6. Use Kruskal’s algorithm to find a minimum spanning tree 
for the weighted graph in Exercise 2. 


7. Use Kruskal’s algorithm to find a minimum spanning tree 
for the weighted graph in Exercise 3. 


8. Use Kruskal’s algorithm to find a minimum spanning tree 
for the weighted graph in Exercise 4. 
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9. Find a connected weighted simple graph with the fewest 
edges possible that has more than one minimum spanning 
tree. 


10. A minimum spanning forest in a weighted graph is 
a spanning forest with minimal weight. Explain how 
Prim’s and Kruskal’s algorithms can be adapted to con- 
struct minimum spanning forests. 


A maximum spanning tree of a connected weighted undi- 
rected graph is a spanning tree with the largest possible 
weight. 


11. Devise an algorithm similar to Prim’s algorithm for 
constructing a maximum spanning tree of a connected 
weighted graph. 


12. Devise an algorithm similar to Kruskal’s algorithm for 
constructing a maximum spanning tree of a connected 
weighted graph. 

13. Find a maximum spanning tree for the weighted graph in 
Exercise 2. 


14. Find a maximum spanning tree for the weighted graph in 
Exercise 3. 


15. Find a maximum spanning tree for the weighted graph in 
Exercise 4. 


16. Find the second least expensive communications net- 
work connecting the five computer centers in the problem 
posed at the beginning of the section. 


*17. Devise an algorithm for finding the second shortest span- 
ning tree in a connected weighted graph. 


*18. Show that an edge with smallest weight in a connected 
weighted graph must be part of any minimum spanning 
tree. 


19. Show that there is a unique minimum spanning tree in a 
connected weighted graph if the weights of the edges are 
all different. 


20. Suppose that the computer network connecting the cities 
in Figure | must contain a direct link between New York 
and Denver. What other links should be included so that 
there is a link between every two computer centers and 
the cost is minimized? 


21. Find a spanning tree with minimal total weight contain- 
ing the edges {e, i} and {g, k} in the weighted graph in 
Figure 3. 

22. Describe an algorithm for finding a spanning tree with 


minimal weight containing a specified set of edges in a 
connected weighted undirected simple graph. 


23. Express the algorithm devised in Exercise 22 in pseu- 
docode. 
Sollin’s algorithm produces a minimum spanning tree from a 
connected weighted simple graph G = (V, E) by successively 
adding groups of edges. Suppose that the vertices in V are or- 
dered. This produces an ordering of the edges where {uọo, vo} 
precedes {u,,v,} if ug precedes u; or if uy = u; and vy pre- 
cedes v,. The algorithm begins by simultaneously choosing 
the edge of least weight incident to each vertex. The first edge 
in the ordering is taken in the case of ties. This produces a 


graph with no simple circuits, that is, a forest of trees (Ex- 
ercise 24 asks for a proof of this fact). Next, simultaneously 
choose for each tree in the forest the shortest edge between a 
vertex in this tree and a vertex in a different tree. Again the first 
edge in the ordering is chosen in the case of ties. (This pro- 
duces a graph with no simple circuits containing fewer trees 
than were present before this step; see Exercise 24.) Continue 
the process of simultaneously adding edges connecting trees 
until n — 1 edges have been chosen. At this stage a minimum 
spanning tree has been constructed. 


*24, Show that the addition of edges at each stage of Sollin’s 
algorithm produces a forest. 


25. Use Sollin’s algorithm to produce a minimum spanning 
tree for the weighted graph shown in 


a) Figure 1. 
b) Figure 3. 


* 26. Express Sollin’s algorithm in pseudocode. 


** 27. Prove that Sollin’s algorithm produces a minimum span- 


ning tree in a connected undirected weighted graph. 


*28. Show that the first step of Sollin’s algorithm produces a 
forest containing at least [n/2] edges when the input is 
an undirected graph with n vertices. 


* 29, Show that if there are r trees in the forest at some interme- 
diate step of Sollin’s algorithm, then at least [r/2] edges 
are added by the next iteration of the algorithm. 


* 30. Show that when given as input an undirected graph withn 
vertices, no more than |n/2*| trees remain after the first 
step of Sollin’s algorithm has been carried out and the 
second step of the algorithm has been carried out k — 1 
times. 


* 31. Show that Sollin’s algorithm requires at most log n iter- 
ations to produce a minimum spanning tree from a con- 
nected undirected weighted graph with n vertices. 


32. Prove that Kruskal’s algorithm produces minimum span- 
ning trees. 


33. Show that if G is a weighted graph with distinct edge 
weights, then for every simple circuit of G, the edge of 
maximum weight in this circuit does not belong to any 
minimum spanning tree of G. 


When Kruskal invented the algorithm that finds minimum 
spanning trees by adding edges in order of increasing weight 
as long as they do not form a simple circuit, he also invented 
another algorithm sometimes called the reverse-delete al- 
gorithm. This algorithm proceeds by successively deleting 
edges of maximum weight from a connected graph as long as 
doing so does not disconnect the graph. 


34. Express the reverse-delete algorithm in pseudocode. 


35. Prove that the reverse-delete algorithm always pro- 
duces a minimum spanning tree when given as input a 
weighted graph with distinct edge weights. [Hint: Use 
Exercise 33.] 
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tree: a connected undirected graph with no simple circuits 
forest: an undirected graph with no simple circuits 


rooted tree: a directed graph with a specified vertex, called the 
root, such that there is a unique path to every other vertex 
from this root 


subtree: a subgraph of a tree that is also a tree 


parent of v in a rooted tree: the vertex u such that (u, v) is an 
edge of the rooted tree 


child of a vertex v in a rooted tree: any vertex with v as its 
parent 


sibling of a vertex v in a rooted tree: a vertex with the same 
parent as v 


ancestor of a vertex v in a rooted tree: any vertex on the path 
from the root to v 


descendant of a vertex v in a rooted tree: any vertex that has 
v as an ancestor 


internal vertex: a vertex that has children 
leaf: a vertex with no children 


level of a vertex: the length of the path from the root to this 
vertex 


height of a tree: the largest level of the vertices of a tree 


m-ary tree: a tree with the property that every internal vertex 
has no more than m children 


full m-ary tree: a tree with the property that every internal 
vertex has exactly m children 


binary tree: an m-ary tree with m = 2 (each child may be des- 
ignated as a left or a right child of its parent) 


ordered tree: a tree in which the children of each internal ver- 
tex are linearly ordered 


balanced tree: a tree in which every leaf is at level h or h — 1, 
where A is the height of the tree 


binary search tree: a binary tree in which the vertices are la- 
beled with items so that a label of a vertex is greater than 
the labels of all vertices in the left subtree of this vertex and 
is less than the labels of all vertices in the right subtree of 
this vertex 


decision tree: a rooted tree where each vertex represents a pos- 
sible outcome of a decision and the leaves represent the pos- 
sible solutions of a problem 


game tree: a rooted tree where vertices represent the possi- 
ble positions of a game as it progresses and edges represent 
legal moves between these positions 


prefix code: a code that has the property that the code of a 
character is never a prefix of the code of another character 


minmax strategy: the strategy where the first player and sec- 
ond player move to positions represented by a child with 
maximum and minimum value, respectively 
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value of a vertex in a game tree: for a leaf, the payoff to the 
first player when the game terminates in the position repre- 
sented by this leaf; for an internal vertex, the maximum or 
minimum of the values of its children, for an internal vertex 
at an even or odd level, respectively 


tree traversal: a listing of the vertices of a tree 


preorder traversal: a listing of the vertices of an ordered 
rooted tree defined recursively—the root is listed, followed 
by the first subtree, followed by the other in the order they 
occur from left to right 


inorder traversal: a listing of the vertices of an ordered rooted 
tree defined recursively—the first subtree is listed, followed 
by the root, followed by the other subtrees in the order they 
occur from left to right 


postorder traversal: a listing of the vertices of an ordered 
rooted tree defined recursively—the subtrees are listed in 
the order they occur from left to right, followed by the 
root 


infix notation: the form of an expression (including a full set 
of parentheses) obtained from an inorder traversal of the 
binary tree representing this expression 


prefix (or Polish) notation: the form of an expression ob- 
tained from a preorder traversal of the tree representing this 
expression 


postfix (or reverse Polish) notation: the form of an expres- 
sion obtained from a postorder traversal of the tree repre- 
senting this expression 


spanning tree: a tree containing all vertices of a graph 


minimum spanning tree: a spanning tree with smallest pos- 
sible sum of weights of its edges 


RESULTS 


A graph is a tree if and only if there is a unique simple path 
between every pair of its vertices. 

A tree with n vertices has n — 1 edges. 

A full m-ary tree with i internal vertices has mi + 1 vertices. 

The relationships among the numbers of vertices, leaves, and 
internal vertices in a full m-ary tree (see Theorem 4 in Sec- 
tion 11.1) 

There are at most m” leaves in an m-ary tree of height h. 

If an m-ary tree has / leaves, its height A is at least [log,,, /]. If 
the tree is also full and balanced, then its height is [log,,, /]. 


Huffman coding: a procedure for constructing an optimal bi- 
nary code for a set of symbols, given the frequencies of 
these symbols 


depth-first search, or backtracking: a procedure for con- 
structing a spanning tree by adding edges that form a path 
until this is not possible, and then moving back up the path 
until a vertex is found where a new path can be formed 
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breadth-first search: a procedure for constructing a spanning 
tree that successively adds all edges incident to the last set 
of edges added, unless a simple circuit is formed 


Prim’s algorithm: a procedure for producing a minimum 
spanning tree in a weighted graph that successively adds 
edges with minimal weight among all edges incident to a 


Review Questions 


1. a) Define a tree. b) Define a forest. 


2. Can there be two different simple paths between the ver- 
tices of a tree? 


3. Give at least three examples of how trees are used in mod- 
eling. 
4. a) Define a rooted tree and the root of such a tree. 

b) Define the parent of a vertex and a child of a vertex in 
a rooted tree. 

c) What are an internal vertex, a leaf, and a subtree in a 
rooted tree? 

d) Draw a rooted tree with at least 10 vertices, where the 
degree of each vertex does not exceed 3. Identify the 
root, the parent of each vertex, the children of each 
vertex, the internal vertices, and the leaves. 


5. a) How many edges does a tree with n vertices have? 
b) What do you need to know to determine the number 
of edges in a forest with n vertices? 


6. a) Define a full m-ary tree. 

b) How many vertices does a full m-ary tree have if it 
has i internal vertices? How many leaves does the tree 
have? 

7. a) What is the height of a rooted tree? 

b) What is a balanced tree? 

c) How many leaves can an m-ary tree of height h have? 
8. a) What is a binary search tree? 

b) Describe an algorithm for constructing a binary 
search tree. 

c) Forma binary search tree for the words vireo, warbler, 
egret, grosbeak, nuthatch, and kingfisher. 

9. a) What is a prefix code? 
b) How can a prefix code be represented by a binary tree? 


10. a) Define preorder, inorder, and postorder tree traversal. 
b) Give an example of preorder, postorder, and inorder 
traversal of a binary tree of your choice with at 

least 12 vertices. 

11. a) Explain how to use preorder, inorder, and postorder 
traversals to find the prefix, infix, and postfix forms of 
an arithmetic expression. 

b) Draw the ordered rooted tree that represents 


(Œ -= 3) + (@/4) + @—y) f 3). 


vertex already in the tree so that no edge produces a simple 
circuit when it is added 

Kruskal’s algorithm: a procedure for producing a mini- 
mum spanning tree in a weighted graph that successively 
adds edges of least weight that are not already in the tree 
such that no edge produces a simple circuit when it is 
added 


c) Find the prefix and postfix forms of the expression in 
part (b). 
12. Show that the number of comparisons used by a sorting 
algorithm to sort a list of n elements is at least [log n!]. 


13. a) Describe the Huffman coding algorithm for construct- 
ing an optimal code for a set of symbols, given the 
frequency of these symbols. 

b) Use Huffman coding to find an optimal code for 
these symbols and frequencies: A: 0.2, B: 0.1, C: 0.3, 
D: 0.4. 


14. Draw the game tree for nim if the starting position con- 
sists of two piles with one and four stones, respectively. 
Who wins the game if both players follow an optimal 
strategy? 


15. a) What is a spanning tree of a simple graph? 
b) Which simple graphs have spanning trees? 
c) Describe at least two different applications that re- 
quire that a spanning tree of a simple graph be found. 


16. a) Describe two different algorithms for finding a span- 
ning tree in a simple graph. 

b) Illustrate how the two algorithms you described in 
part (a) can be used to find the spanning tree of a sim- 
ple graph, using a graph of your choice with at least 
eight vertices and 15 edges. 


17. a) Explain how backtracking can be used to determine 
whether a simple graph can be colored using n colors. 

b) Show, with an example, how backtracking can be used 
to show that a graph with a chromatic number equal 
to 4 cannot be colored with three colors, but can be 
colored with four colors. 

18. a) What is a minimum spanning tree of a connected 
weighted graph? 

b) Describe at least two different applications that re- 
quire that a minimum spanning tree of a connected 
weighted graph be found. 

19. a) Describe Kruskal’s algorithm and Prim’s algorithm 
for finding minimum spanning trees. 

b) Illustrate how Kruskal’s algorithm and Prim’s algo- 
rithm are used to find a minimum spanning tree, us- 
ing a weighted graph with at least eight vertices and 
15 edges. 


Supplementary Exercises 


* 1. Show that a simple graph is a tree if and only if it contains 
no simple circuits and the addition of an edge connecting 
two nonadjacent vertices produces a new graph that has 
exactly one simple circuit (where circuits that contain the 
same edges are not considered different). 


* 2, How many nonisomorphic rooted trees are there with six 
vertices? 


3. Show that every tree with at least one edge must have at 
least two pendant vertices. 


4. Show that a tree with n vertices that has n — 1 pendant 
vertices must be isomorphic to Ky, ,,_;. 


5. What is the sum of the degrees of the vertices of a tree 
with n vertices? 


*6, Suppose that d,,d,,...,d, are n positive integers with 
sum 2n — 2. Show that there is a tree that has n vertices 
such that the degrees of these vertices are d}, dy, ..., d. 

7. Show that every tree is a planar graph. 

8. Show that every tree is bipartite. 


9. Show that every forest can be colored using two colors. 


A B-tree of degree k is a rooted tree such that all its leaves 
are at the same level, its root has at least two and at most 
k children unless it is a leaf, and every internal vertex other 
than the root has at least [k/2], but no more than k, children. 
Computer files can be accessed efficiently when B-trees are 
used to represent them. 


10. Draw three different B-trees of degree 3 with height 4. 


*11. Give an upper bound and a lower bound for the number 
of leaves in a B-tree of degree k with height h. 


*12. Give an upper bound and a lower bound for the height of 
a B-tree of degree k with n leaves. 


The binomial trees B,, i = 0, 1, 2, ..., are ordered rooted trees 
defined recursively: 


Basis step: The binomial tree By is the tree with a single 
vertex. 


Recursive step: Let k be a nonnegative integer. To con- 
struct the binomial tree B,,,, add a copy of B, to a second 
copy of B, by adding an edge that makes the root of the 
first copy of B, the leftmost child of the root of the second 
copy of B,. 


13. Draw B, for k = 0, 1, 2, 3, 4. 


14. How many vertices does B, have? Prove that your answer 
is correct. 


15. Find the height of B,. Prove that your answer is correct. 
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16. How many vertices are there in B, at depth j, where 
0 <j < k? Justify your answer. 


17. What is the degree of the root of B,? Prove that your an- 
swer is correct. 


18. Show that the vertex of largest degree in B, is the root. 


A rooted tree T is called an S,-tree if it satisfies this recursive 
definition. It is an Sp-tree if it has one vertex. For k > 0, T is 
an S,-tree if it can be built from two S,_,-trees by making the 
root of one the root of the S,-tree and making the root of the 
other the child of the root of the first S,_;-tree. 


19. Draw an S,-tree for k = 0, 1, 2, 3, 4. 


20. Show that an S,-tree has 2* vertices and a unique vertex 
at level k. This vertex at level k is called the handle. 


*21. Suppose that T is an S,-tree with handle v. Show 
that T can be obtained from disjoint trees Tọ, T}, ..., T,_1, 
with roots rọ, F1, ..., 7-1, respectively, where v is not 
in any of these trees, where 7; is an S,-tree for i= 
0, 1,..., k— 1, by connecting v to rọ and r; to 7;,, for 
i=0,1,...,4—-2. 

The listing of the vertices of an ordered rooted tree in level 
order begins with the root, followed by the vertices at level 1 
from left to right, followed by the vertices at level 2 from left 
to right, and so on. 


22. List the vertices of the ordered rooted trees in Figures 3 
and 9 of Section 11.3 in level order. 


23. Devise an algorithm for listing the vertices of an ordered 
rooted tree in level order. 


*24. Devise an algorithm for determining if a set of universal 
addresses can be the addresses of the leaves of a rooted 
tree. 


25. Devise an algorithm for constructing a rooted tree from 
the universal addresses of its leaves. 


A cut set of a graph is a set of edges such that the removal of 
these edges produces a subgraph with more connected com- 
ponents than in the original graph, but no proper subset of this 
set of edges has this property. 


26. Show that a cut set of a graph must have at least one edge 
in common with any spanning tree of this graph. 


A cactus is a connected graph in which no edge is in more 
than one simple circuit not passing through any vertex other 
than its initial vertex more than once or its initial vertex 
other than at its terminal vertex (where two circuits that con- 
tain the same edges are not considered different). 
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27. Which of these graphs are cacti? 
a) 


28. Is a tree necessarily a cactus? 


29. Show that a cactus is formed if we add a circuit contain- 
ing new edges beginning and ending at a vertex of a tree. 


* 30. Show that if every circuit not passing through any vertex 
other than its initial vertex more than once in a connected 
graph contains an odd number of edges, then this graph 
must be a cactus. 

A degree-constrained spanning tree of a simple graph G 

is a spanning tree with the property that the degree of a ver- 

tex in this tree cannot exceed some specified bound. Degree- 
constrained spanning trees are useful in models of transporta- 
tion systems where the number of roads at an intersection is 
limited, models of communications networks where the num- 
ber of links entering a node is limited, and so on. 

In Exercises 31—33 find a degree-constrained spanning 
tree of the given graph where each vertex has degree less than 
or equal to 3, or show that such a spanning tree does not exist. 


31. 


32. b 


a C 


34. Show that a degree-constrained spanning tree of a simple 
graph in which each vertex has degree not exceeding 2 
consists of a single Hamilton path in the graph. 


35. A tree with n vertices is called graceful if its vertices 
can be labeled with the integers 1, 2, ..., n such that the 
absolute values of the difference of the labels of adja- 
cent vertices are all different. Show that these trees are 
graceful. 

a) “—*—*—* 


A caterpillar is a tree that contains a simple path such that 
every vertex not contained in this path is adjacent to a vertex 
in the path. 

36. Which of the graphs in Exercise 35 are caterpillars? 


37. How many nonisomorphic caterpillars are there with six 
vertices? 


** 38. a) Prove or disprove that all trees whose edges form a 


single path are graceful. 
b) Prove or disprove that all caterpillars are graceful. 


39. Suppose that in a long bit string the frequency of occur- 
rence of a 0 bit is 0.9 and the frequency of a 1 bit is 0.1 
and bits occur independently. 

a) Construct a Huffman code for the four blocks of two 
bits, 00, 01, 10, and 11. What is the average number 
of bits required to encode a bit string using this code? 

b) Construct a Huffman code for the eight blocks of three 
bits. What is the average number of bits required to 
encode a bit string using this code? 

40. Suppose that G is a directed graph with no circuits. De- 
scribe how depth-first search can be used to carry out a 
topological sort of the vertices of G. 


*41. Suppose that e is an edge in a weighted graph that is in- 


cident to a vertex v such that the weight of e does not 
exceed the weight of any other edge incident to v. Show 
that there exists a minimum spanning tree containing this 
edge. 

42. Three couples arrive at the bank of a river. Each of the 
wives is jealous and does not trust her husband when he 
is with one of the other wives (and perhaps with other 
people), but not with her. How can six people cross to the 
other side of the river using a boat that can hold no more 
than two people so that no husband is alone with a woman 
other than his wife? Use a graph theory model. 


* 43. 


44. 


Show that if no two edges in a weighted graph have the 
same weight, then the edge with least weight incident to 
a vertex v is included in every minimum spanning tree. 


Find a minimum spanning tree of each of these graphs 
where the degree of each vertex in the spanning tree does 
not exceed 2. 


Let G = (V, E) be a directed graph and let r be a vertex in G. 
An arborescence of G rooted at r is a subgraph T = (V, F) of 
G such that the underlying undirected graph of T is a spanning 


Computer Projects 


Write programs with these input and output. 


10. 


11. 


Given the adjacency matrix of an undirected simple 
graph, determine whether the graph is a tree. 


Given the adjacency matrix of a rooted tree and a vertex 
in the tree, find the parent, children, ancestors, descen- 
dants, and level of this vertex. 


Given the list of edges of a rooted tree and a vertex in the 
tree, find the parent, children, ancestors, descendants, and 
level of this vertex. 


Given a list of items, construct a binary search tree con- 
taining these items. 

Given a binary search tree and an item, locate or add this 
item to the binary search tree. 

Given the ordered list of edges of an ordered rooted tree, 
find the universal addresses of its vertices. 

Given the ordered list of edges of an ordered rooted tree, 
list its vertices in preorder, inorder, and postorder. 
Given an arithmetic expression in prefix form, find its 
value. 


Given an arithmetic expression in postfix form, find its 
value. 


Given the frequency of symbols, use Huffman coding to 
find an optimal code for these symbols. 


Given an initial position in the game of nim, determine 
an optimal strategy for the first player. 
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tree of the underlying undirected graph of G and for every ver- 
tex v € V there is a path from r to v in T (with directions taken 
into account). 


45. 


46. 


47. 


12. 


13. 


14. 


*15, 


*16. 


17. 


18. 


Show that a subgraph T = (V, F) of the graph G = (V, E) 
is an arborescence of G rooted at r if and only if T con- 
tains r, T has no simple circuits, and for every vertex 
v E V other than r, deg (v) = 1 in T. 

Show that a directed graph G = (V, E) has an arbores- 

cence rooted at the vertex r if and only if for every vertex 

v E V, there is a directed path from r to v. 

In this exercise we will develop an algorithm to find the 

strong components of a directed graph G = (V, E). Recall 

that a vertex w € V is reachable from a vertex v € V if 

there is a directed path from v to w. 

a) Explain how to use breadth-first search in the directed 
graph G to find all the vertices reachable from a vertex 
vEG. 

b) Explain how to use breadth-first search in G°” to find 
all the vertices from which a vertex v € Gis reachable. 
(Recall that G° is the directed graph obtained from 
G by reversing the direction of all its edges.) 

c) Explain how to use parts (a) and (b) to construct an al- 
gorithm that finds the strong components of a directed 
graph G, and explain why your algorithm is correct. 


Given the adjacency matrix of a connected undirected 
simple graph, find a spanning tree for this graph using 
depth-first search. 


Given the adjacency matrix of a connected undirected 
simple graph, find a spanning tree for this graph using 
breadth-first search. 


Given a set of positive integers and a positive integer N, 
use backtracking to find a subset of these integers that 
have N as their sum. 


Given the adjacency matrix of an undirected simple 
graph, use backtracking to color the graph with three col- 
ors, if this is possible. 


Given a positive integer n, solve the n-queens problem 
using backtracking. 


Given the list of edges and their weights of a weighted 
undirected connected graph, use Prim’s algorithm to find 
a minimum spanning tree of this graph. 


Given the list of edges and their weights of a weighted 
undirected connected graph, use Kruskal’s algorithm to 
find a minimum spanning tree of this graph. 
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Computations and Explorations 


Use a computational program or programs you have written to do these exercises. 


*3, 


Display all trees with six vertices. 


. Display a full set of nonisomorphic trees with seven ver- 


tices. 


Construct a Huffman code for the symbols with ASCII 
codes given the frequency of their occurrence in repre- 
sentative input. 

Compute the number of different spanning trees of K, for 
n= 1, 2, 3, 4,5, 6. Conjecture a formula for the number of 
such spanning trees whenever n is a positive integer. 
Compare the number of comparisons needed to sort lists 
of n elements for n = 100, 1000, and 10,000 from the set 
of positive integers less than 1,000,000, where the ele- 
ments are randomly selected positive integers, using the 


Writing Projects 


Respond to these with essays using outside sources. 


. Explain how Cayley used trees to enumerate the number 


of certain types of hydrocarbons. 


. Explain how trees are used to represent ancestral relations 


in the study of evolution. 


. Discuss hierarchical cluster trees and how they are used. 
. Define AVL-trees (sometimes also known as height- 


balanced trees). Describe how and why AVL-trees are 
used in a variety of different algorithms. 


. Define quad trees and explain how images can be repre- 


sented using them. Describe how images can be rotated, 
scaled, and translated by manipulating the corresponding 
quad tree. 


. Define a heap and explain how trees can be turned into 


heaps. Why are heaps useful in sorting? 


. Describe dynamic algorithms for data compression based 


on letter frequencies as they change as characters are suc- 
cessively read, such as adaptive Huffman coding. 


. Explain how alpha-beta pruning can be used to simplify 


the computation of the value of a game tree. 


. Describe the techniques used by chess-playing programs 


such as Deep Blue. 


*7, 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


selection sort, the insertion sort, the merge sort, and the 
quick sort. 


. Compute the number of different ways n queens can be 


arranged on an n Xn chessboard so that no two queens 
can attack each other for all positive integers n not 
exceeding 10. 


Find a minimum spanning tree of the graph that connects 
the capital cities of the 50 states in the United States to 
each other where the weight of each edge is the distance 
between the cities. 


. Draw the complete game tree for a game of checkers on a 


4 x 4 board. 


Define the type of graph known as a mesh of trees. Ex- 
plain how this graph is used in applications to very large 
system integration and parallel computing. 

Discuss the algorithms used in IP multicasting to avoid 
loops between routers. 

Describe an algorithm based on depth-first search for 
finding the articulation points of a graph. 

Describe an algorithm based on depth-first search to find 
the strongly connected components of a directed graph. 
Describe the search techniques used by Googlebot, and 
other web crawlers and spiders used by different web 
search engines. 

Describe an algorithm for finding the minimum spanning 
tree of a graph such that the maximum degree of any ver- 
tex in the spanning tree does not exceed a fixed constant k. 
Compare and contrast some of the most important sorting 
algorithms in terms of their complexity and when they are 
used. 

Discuss the history and origins of algorithms for con- 
structing minimum spanning trees. 


Describe algorithms for producing random trees. 
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Boolean Algebra 


T he circuits in computers and other electronic devices have inputs, each of which is either 
a 0 ora 1, and produce outputs that are also Os and 1s. Circuits can be constructed using 
any basic element that has two different states. Such elements include switches that can be 
in either the on or the off position and optical devices that can be either lit or unlit. In 1938 
Claude Shannon showed how the basic rules of logic, first given by George Boole in 1854 in his 
The Laws of Thought, could be used to design circuits. These rules form the basis for Boolean 
algebra. In this chapter we develop the basic properties of Boolean algebra. The operation of a 
circuit is defined by a Boolean function that specifies the value of an output for each set of inputs. 
The first step in constructing a circuit is to represent its Boolean function by an expression built 
up using the basic operations of Boolean algebra. We will provide an algorithm for producing 
such expressions. The expression that we obtain may contain many more operations than are 
necessary to represent the function. Later in the chapter we will describe methods for finding an 
expression with the minimum number of sums and products that represents a Boolean function. 
The procedures that we will develop, Karnaugh maps and the Quine—McCluskey method, are 
important in the design of efficient circuits. 


Boolean Functions 


EXAMPLE 1 


12.1.1 Introduction 


Boolean algebra provides the operations and the rules for working with the set {0, 1}. Electronic 
and optical switches can be studied using this set and the rules of Boolean algebra. The three 
operations in Boolean algebra that we will use most are complementation, the Boolean sum, 
and the Boolean product. The complement of an element, denoted with a bar, is defined by 


0 = 1 and 1 = 0. The Boolean sum, denoted by + or by OR, has the following values: 
1+1=1, 1+0=1, O+1=1, 0+0=0. 

The Boolean product, denoted by - or by AND, has the following values: 

1-1=1, 1-0=0, 0-1=0, 0-0=0. 


When there is no danger of confusion, the symbol - can be deleted, just as in writing algebraic 
products. Unless parentheses are used, the rules of precedence for Boolean operators are: first, 
all complements are computed, followed by all Boolean products, followed by all Boolean sums. 
This is illustrated in Example 1. 


Find the value of 1-0+(0+4 1). 


Solution: Using the definitions of complementation, the Boolean sum, and the Boolean product, 
it follows that 


1-0+(0+1)=0+1 
=0+0 
=0. < 
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EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 
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The complement, Boolean sum, and Boolean product correspond to the logical operators, 
=, V, and A, respectively, where 0 corresponds to F (false) and 1 corresponds to T (true). Equal- 
ities in Boolean algebra can be directly translated into equivalences of compound propositions. 
Conversely, equivalences of compound propositions can be translated into equalities in Boolean 
algebra. We will see later in this section why these translations yield valid logical equivalences 
and identities in Boolean algebra. Example 2 illustrates the translation from Boolean algebra to 
propositional logic. 


Translate 1 - 0 + (0 + 1) = 0, the equality found in Example 1, into a logical equivalence. 


Solution: We obtain a logical equivalence when we translate each 1 into a T, each 0 into an F, 
each Boolean sum into a disjunction, each Boolean product into a conjunction, and each com- 
plementation into a negation. We obtain 


(TAF) V-(T VF) =F. < 


Example 3 illustrates the translation from propositional logic to Boolean algebra. 


Translate the logical equivalence (T A T) v ~F = T into an identity in Boolean algebra. 


Solution: We obtain an identity in Boolean algebra when we translate each T into a 1, each F 
into a 0, each disjunction into a Boolean sum, each conjunction into a Boolean product, and 
each negation into a complementation. We obtain 


(-1)+0=1. < 


12.1.2 Boolean Expressions and Boolean Functions 


Let B = {0,1}. Then B” = {(x;, x5, ...,x,) |x; E B for 1 <i <n} is the set of all possible 
n-tuples of Os and 1s. The variable x is called a Boolean variable if it assumes values only 
from B, that is, if its only possible values are 0 and 1. A function from B” to B is called a 
Boolean function of degree n. 


The function F(x, y) = xy from the set of ordered pairs of Boolean variables to the set {0, 1} is 
a Boolean function of degree 2 with F(1, 1) = 0, FC, 0) = 1, F(O, 1) = 0, and F(0, 0) = 0. We 
display these values of F in Table 1. < 


CLAUDE ELWOOD SHANNON (1916-2001) Claude Shannon was born in Petoskey, Michigan, and grew up 
in Gaylord, Michigan. His father was a businessman and a probate judge, and his mother was a language teacher 
and a high school principal. Shannon attended the University of Michigan, graduating in 1936. He continued 
his studies at M.I.T., where he took the job of maintaining the differential analyzer, a mechanical computing 
device consisting of shafts and gears built by his professor, Vannevar Bush. Shannon’s master’s thesis, written in 
1936, studied the logical aspects of the differential analyzer. This master’s thesis presents the first application of 
Boolean algebra to the design of switching circuits; it is perhaps the most famous master’s thesis of the twentieth 
century. He received his Ph.D. from M.LT. in 1940. Shannon joined Bell Laboratories in 1940, where he worked 
on transmitting data efficiently. He was one of the first people to use bits to represent information. At Bell 
Laboratories he worked on determining the amount of traffic that telephone lines can carry. Shannon made many 
fundamental contributions to information theory. In the early 1950s he was one of the founders of the study of 
artificial intelligence. He joined the M.LT. faculty in 1956, where he continued his study of information theory. 


Shannon had an unconventional side. He is credited with inventing the rocket-powered Frisbee. He is also famous for riding a 
unicycle down the hallways of Bell Laboratories while juggling four balls. Shannon retired when he was 50 years old, publishing 
papers sporadically over the following 10 years. In his later years he concentrated on some pet projects, such as building a motorized 
pogo stick. One interesting quote from Shannon, published in Omni Magazine in 1987, is “I visualize a time when we will be to 
robots what dogs are to humans. And I am rooting for the machines.” 
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Boolean functions can be represented using expressions made up from variables and 


Boolean operations. The Boolean expressions in the variables x,, X2, ..., X, are defined recur- 
sively as 
0, 1, x1, X2, ..., x, are Boolean expressions; 
if E; and E, are Boolean expressions, then E,, (E,£,), and (E; + E,) are Boolean 
expressions. 


Each Boolean expression represents a Boolean function. The values of this function are obtained 
by substituting 0 and 1 for the variables in the expression. In Section 12.2 we will show that 
every Boolean function can be represented by a Boolean expression. 


Find the values of the Boolean function represented by F(x, y, z) = xy + Z. 


Solution: The values of this function are displayed in Table 2. < 


TABLE 2 


1 
1 
1 
1 
0 
0 
0 
0 


ea 
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Note that we can represent a Boolean function graphically by distinguishing the vertices of 
the n-cube that correspond to the n-tuples of bits where the function has value 1. 


The function F(x, y, z) = xy +Z from B? to B from Example 5 can be represented by distin- 
guishing the vertices that correspond to the five 3-tuples (1, 1, 1), (1, 1, 0), (1, 0, 0), (0, 1, 0), and 
(0, 0, 0), where F(x, y, z) = 1, as shown in Figure 1. These vertices are displayed using solid 
black circles. < 


Boolean functions F and G of n variables are equal if and only if F(b}, bz, ..., b,) = 
G(b,, bz, ..., b,,) whenever b,, bz, ..., b, belong to B. Two different Boolean expressions that 
represent the same function are called equivalent. For instance, the Boolean expressions xy, 
xy + 0, and xy - 1 are equivalent. The complement of the Boolean function F is the function F, 
where F(x, ...,X,) = FŒ ...,%,). Let F and G be Boolean functions of degree n. The Boolean 
sum F + G and the Boolean product FG are defined by 


(F + G)&p 00, Xp) = Fy 00, Xp) + GÆ 0 Xp), 
(FG) &p 0, %,) = Fp 0. %,)GHp 0 X). 


A Boolean function of degree two is a function from a set with four elements, namely, pairs 
of elements from B = {0, 1}, to B, a set with two elements. Hence, there are 16 different Boolean 
functions of degree two. In Table 3 we display the values of the 16 different Boolean functions 
of degree two, labeled F}, Fy, ..., F16- 


EXAMPLE 7 


EXAMPLE 8 
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TABLE 3 The 16 Boolean Functions of Degree Two. 
1} 1 1 


How many different Boolean functions of degree n are there? 


Solution: From the product rule for counting, it follows that there are 2” different n-tuples 
of Os and 1s. Because a Boolean function is an assignment of 0 or 1 to each of these 2” 
different n-tuples, the product rule shows that there are 2?" different Boolean functions of 
degree n. < 


Table 4 displays the number of different Boolean functions of degrees one through six. The 
number of such functions grows extremely rapidly. 


TABLE 4 The Number of Boolean 
Functions of Degree n. 


256 

65,536 

4,294,967,296 
18,446,744,073,709,551,616 


12.1.3 Identities of Boolean Algebra 


There are many identities in Boolean algebra. The most important of these are displayed in 
Table 5. These identities are particularly useful in simplifying the design of circuits. Each of 
the identities in Table 5 can be proved using a table. We will prove one of the distributive laws 
in this way in Example 8. The proofs of the remaining properties are left as exercises for the 
reader. 


Show that the distributive law x(y + z) = xy + xz is valid. 


Solution: The verification of this identity is shown in Table 6. The identity holds because the 
last two columns of the table agree. < 


The reader should compare the Boolean identities in Table 5 to the logical equivalences in 
Table 6 of Section 1.3 and the set identities in Table 1 in Section 2.2. All are special cases of the 
same Set of identities in a more abstract structure. Each collection of identities can be obtained 
by making the appropriate translations. For example, we can transform each of the identities in 
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TABLE 5 Boolean Identities. 


Law of the double complement 


Idempotent laws 


Identity laws 


Domination laws 


Compare these Boolean 

identities with the y= Commutative laws 

logical equivalences in 

Section 1.3 and the set 

identities in Section 2.2. x+ (y +z)=(x+y)+z Associative laws 
xz) = (xy)z 


x+yz=(x+yX(x+z) Distributive laws 
x +z) = xy + xz 


De Morgan’s laws 


Absorption laws 


Table 5 into a logical equivalence by changing each Boolean variable into a propositional vari- 
able, each 0 into a F, each 1 into a T, each Boolean sum into a disjunction, each Boolean product 
into a conjunction, and each complementation into a negation, as we illustrate in Example 9. 


EXAMPLE 9 Translate the distributive law x + yz = (x + y)(x + z) in Table 5 into a logical equivalence. 
Solution: To translate a Boolean identity into a logical equivalence, we change each Boolean 


variable into a propositional variable. Here we will change the Boolean variables x, y, and z into 
the propositional variables p, q, and r. Next, we change each Boolean sum into a disjunction and 


TABLE 6 Verifying One of the Distributive Laws. 


E 
a 
z 
a 
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N 
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1 
1 
1 
1 
0 
0 
0 
0 
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EXAMPLE 10 


Extra » 
Examples 


EXAMPLE 11 


EXAMPLE 12 


each Boolean product into a conjunction. (Note that 0 and 1 do not appear in this identity and 
complementation also does not appear.) This transforms the Boolean identity into the logical 
equivalence 


PVGAN) = PVQDA(PV?). 
This logical equivalence is one of the distributive laws for propositional logic in Table 6 in 
Section 1.3. < 
Identities in Boolean algebra can be used to prove further identities. We demonstrate this 


in Example 10. 


Prove the absorption law x(x + y) = x using the other identities of Boolean algebra shown in 
Table 5. (This is called an absorption law because absorbing x + y into x leaves x unchanged.) 


Solution: We display steps used to derive this identity and the law used in each step: 


x(x + y) = (x+ 0)(x +y) Identity law for the Boolean sum 


=x+0-y Distributive law of the Boolean sum over the 
Boolean product 
=xt+y-0 Commutative law for the Boolean product 
=x+0 Domination law for the Boolean product 
=X Identity law for the Boolean sum. < 


12.1.4 Duality 


The identities in Table 5 come in pairs (except for the law of the double complement and the 
unit and zero properties). To explain the relationship between the two identities in each pair 
we use the concept of a dual. The dual of a Boolean expression is obtained by interchanging 
Boolean sums and Boolean products and interchanging Os and 1s. 


Find the duals of x(y + 0) and x - 1 + O + 2). 


Solution: Interchanging - signs and + signs and interchanging Os and 1s in these expressions 
produces their duals. The duals are x + (y - 1) and (x + 0)(yz), respectively. < 


The dual of a Boolean function F represented by a Boolean expression is the function repre- 
sented by the dual of this expression. This dual function, denoted by F, does not depend on the 
particular Boolean expression used to represent F. An identity between functions represented 
by Boolean expressions remains valid when the duals of both sides of the identity are taken. 
(See Exercise 30 for the reason why this is true.) This result, called the duality principle, is 
useful for obtaining new identities. 


Construct an identity from the absorption law x(x + y) = x by taking duals. 


Solution: Taking the duals of both sides of this identity produces the identity x + xy = x, which 
is also called an absorption law and is shown in Table 5. < 


Definition 1 


12.1 Boolean Functions 853 


12.1.5 The Abstract Definition of a Boolean Algebra 


In this section we have focused on Boolean functions and expressions. However, the results we 
have established can be translated into results about propositions or results about sets. Because 
of this, it is useful to define Boolean algebras abstractly. Once it is shown that a particular 
structure is a Boolean algebra, then all results established about Boolean algebras in general 
apply to this particular structure. 

Boolean algebras can be defined in several ways. The most common way is to specify the 
properties that operations must satisfy, as is done in Definition 1. 


A Boolean algebra is a set B with two binary operations V and A, elements 0 and 1, and a 
unary operation ~ such that these properties hold for all x, y, and z in B: 


GV Oe : 

oe | Identity laws 

TEN) Se = Il 

ee oy Complement laws 


Associative laws 


CNN A ANVON) 
(XAY)AZ=XAWAZ) 
XV yy a Commutative laws 
XAVY=YVAX 
XV(VAZ)=(&VY)AV2Z) 


XA(YVZ)=(LAY)V(KA 7 Distributive laws 


Using the laws given in Definition 1, it is possible to prove many other laws that hold for every 
Boolean algebra, such as idempotent and domination laws. (See Exercises 35—42.) 

From our previous discussion, B = {0, 1} with the OR and AND operations and the com- 
plement operator, satisfies all these properties. The set of propositions in n variables, with the V 
and A operators, F and T, and the negation operator, also satisfies all the properties of a Boolean 
algebra, as can be seen from Table 6 in Section 1.3. Similarly, the set of subsets of a universal 
set U with the union and intersection operations, the empty set and the universal set, and the 
set complementation operator, is a Boolean algebra as can be seen by consulting Table 1 in 
Section 2.2. So, to establish results about each of Boolean expressions, propositions, and sets, 
we need only prove results about abstract Boolean algebras. 

Boolean algebras may also be defined using the notion of a lattice, discussed in Chapter 9. 
Recall that a lattice L is a partially ordered set in which every pair of elements x, y has a least 
upper bound, denoted by lub(x, y) and a greatest lower bound denoted by glb(x, y). Given two 
elements x and y of L, we can define two operations V and A on pairs of elements of L by 
x Vy = lub(x, y) and x A y = glb(x, y). 

For a lattice L to be a Boolean algebra as specified in Definition 1, it must have two prop- 
erties. First, it must be complemented. For a lattice to be complemented it must have a least 
element 0 and a greatest element 1 and for every element x of the lattice there must exist an el- 
ement x such that x V x = 1 and x Ax = 0. Second, it must be distributive. This means that for 
every x, y, and z in L, x V (y Az) = (x V y) A (xV z) and x A (WV z) = (AY) V (XA z). Showing 
that a complemented, distributive lattice is a Boolean algebra has been left as Supplementary 
Exercise 39 in Chapter 9. 
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Exercises 


ES 12 


13. 


. Find the values of these expressions. 


a) 1-0 b) 141 90.0 d) (+0) 


. Find the values, if any, of the Boolean variable x that sat- 


isfy these equations. 
a) x-1=0 
c) x-l=x 


b) x+x=0 
d) x-x=1 


. a) Show that (1-1) + (0-140) =1. 


b) Translate the equation in part (a) into a propositional 
equivalence by changing each O into an F, each 1 
into a T, each Boolean sum into a disjunction, each 
Boolean product into a conjunction, each comple- 
mentation into a negation, and the equals sign into a 
propositional equivalence sign. 


. a) Show that (1-0)+ (1-0) = 1. 


b) Translate the equation in part (a) into a propositional 
equivalence by changing each O into an F, each 1 
into a T, each Boolean sum into a disjunction, each 
Boolean product into a conjunction, each comple- 
mentation into a negation, and the equals sign into a 
propositional equivalence sign. 


. Use a table to express the values of each of these Boolean 


functions. 

a) F(x, y, z) = xy 

b) FQ, y, 2) =x+ yz 

© F(x, y, z) = x7 + 02) 
d) F(x, y, z) = xyz + y2) 


. Use a table to express the values of each of these Boolean 


functions. 

a) F(x,y, 2 =Z 

b) FQ, y, z) = xy + yz 

c) F(x, y, z) = xyz + (xyz) 
d) F(x, y, z) = y(xz + xz) 


. Use a 3-cube Q, to represent each of the Boolean func- 


tions in Exercise 5 by displaying a black circle at each 
vertex that corresponds to a 3-tuple where this function 
has the value 1. 


. Use a 3-cube Q, to represent each of the Boolean func- 


tions in Exercise 6 by displaying a black circle at each 
vertex that corresponds to a 3-tuple where this function 
has the value 1. 


. What values of the Boolean variables x and y satisfy 
xy=xt+y? 

. How many different Boolean functions are there of de- 
gree 7? 

. Prove the absorption law x + xy = x using the other laws 
in Table 5. 


. Show that F(x, y, z) = xy + xz + yz has the value 1 if and 


only if at least two of the variables x, y, and z have the 
value 1. 


Show that xy + yZ + Xz = xy + yz + xz. 


Exercises 14-23 deal with the Boolean algebra {0, 1} with ad- 
dition, multiplication, and complement defined at the begin- 
ning of this section. In each case, use a table as in Example 8. 


14. 
15. 
16. 
17. 
18. 
19. 
20. 
21. 
22. 
23. 


Verify the law of the double complement. 
Verify the idempotent laws. 

Verify the identity laws. 

Verify the domination laws. 

Verify the commutative laws. 

Verify the associative laws. 

Verify the first distributive law in Table 5. 
Verify De Morgan’s laws. 

Verify the unit property. 

Verify the zero property. 


The Boolean operator @, called the XOR operator, is defined 
by 1@1=0,1@80=1,0@1=1, and0@0=0. 


24. 


25. 


26. 
27. 


28. 


* 29, 


* 30. 


*31, 


* 32, 


33. 


34. 


Simplify these expressions. 

a) x®0 b) x@l 

c) x@x d) x®x 

Show that these identities hold. 

a) xy = Œ +y) 

b) x®y = QY) + Qy) 

Show that x Q y=y@x. 

Prove or disprove these equalities. 

a) xOVOD=A0y) ®z 

b) x+ 0 @O2=Aty) O@+2) 

ce) xO+2=AOy)+@ Oz) 

Find the duals of these Boolean expressions. 

a) x+y b) xy 

c) xyzt+xyz d) xz+x-0+%X-1 
Suppose that F is a Boolean function represented by a 
Boolean expression in the variables x}, ..., x,,. Show that 
F(x), ...,X,) = F(X, «0.5 X,)- 

Show that if F and G are Boolean functions represented 
by Boolean expressions in n variables and F = G, then 
F? = G’, where F? and Gf are the Boolean functions rep- 
resented by the duals of the Boolean expressions repre- 
senting F and G, respectively. [Hint: Use the result of 
Exercise 29.] 

How many different Boolean functions F(x, y, z) are 
there such that FŒ, y, Z) = F(x, y, z) for all values of the 
Boolean variables x, y, and z? 

How many different Boolean functions F(x, y, z) are there 
such that F(x, y, z) = F(x, y, z) = F(x, y, z ) for all values 
of the Boolean variables x, y, and z? 

Show that you obtain De Morgan’s laws for proposi- 
tions (in Table 6 in Section 1.3) when you transform De 
Morgan’s laws for Boolean algebra in Table 6 into logical 
equivalences. 

Show that you obtain the absorption laws for propositions 
(in Table 6 in Section 1.3) when you transform the ab- 
sorption laws for Boolean algebra in Table 6 into logical 
equivalences. 
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In Exercises 35—42, use the laws in Definition 1 to show that That is, show that for all x and y, (x Vy) =XAy and 

the stated properties hold in every Boolean algebra. (Ay) =XVV. 

35. Show that in a Boolean algebra, the idempotent laws 40. Show that in a Boolean algebra, the modular properties 
xV x =x and x Ax = x hold for every element x. hold. That is, show that x A Y V x Az) =(®Ay)VQ@A 


zyandxV (vA (XV Z)) =AVY)A(KV 2). 
41. Show that in a Boolean algebra, if x V y = 0, then x = 0 
and y = 0, and that if x A y = 1, then x = 1 andy = 1. 


36. Show that in a Boolean algebra, every element x has a 
unique complement x such that x V x = 1 and x Ax = 0. 


37. Show that in a Boolean algebra, the complement of the 42. Show that in a Boolean algebra, the dual of an iden- 
element 0 is the element 1 and vice versa. tity, obtained by interchanging the v and A operators 
and interchanging the elements 0 and 1, is also a valid 


38. Prove that in a Boolean algebra, the law of the double identit 
= identity. 

l holds; that =x fe 1 tx. 
comp ement ee y= ey elementa 43. Show that a complemented, distributive lattice is a 


39. Show that De Morgan’s laws hold in a Boolean algebra. Boolean algebra. 


Representing Boolean Functions 


Two important problems of Boolean algebra will be studied in this section. The first problem 
is: Given the values of a Boolean function, how can a Boolean expression that represents this 
function be found? This problem will be solved by showing that any Boolean function can be 
represented by a Boolean sum of Boolean products of the variables and their complements. The 
solution of this problem shows that every Boolean function can be represented using the three 
Boolean operators -, +, and —. The second problem is: Is there a smaller set of operators that 
can be used to represent all Boolean functions? We will answer this question by showing that 
all Boolean functions can be represented using only one operator. Both of these problems have 
practical importance in circuit design. 


12.2.1 Sum-of-Products Expansions 


We will use examples to illustrate one important way to find a Boolean expression that represents 
a Boolean function. 


EXAMPLE 1 Find Boolean expressions that represent the functions F(x, y, z) and G(x, y, z), which are given 
in Table 1. 


Solution: An expression that has the value 1 when x = z = | and y = 0, and the value 0 other- 
wise, is needed to represent F. Such an expression can be formed by taking the Boolean product 
of x, y, and z. This product, xyz, has the value 1 if and only if x = y = z = 1, which holds if and 
only ifx =z=1landy=0. 

To represent G, we need an expression that equals 1 when x = y = 1 and z = 0, orx =z = 0 
and y = 1. We can form an expression with these values by taking the Boolean sum of two 
different Boolean products. The Boolean product xyz has the value 1 if and only if x = y = 1 
and z = 0. Similarly, the product xyz has the value 1 if and only if x = z = 0 and y = 1. The 
Boolean sum of these two products, xyz + xyz, represents G, because it has the value 1 if and 
only if x = y = 1l and z = 0, or x =z = Qand y= 1. < 


Example 1 illustrates a procedure for constructing a Boolean expression representing a func- 
tion with given values. Each combination of values of the variables for which the function has 
the value 1 leads to a Boolean product of the variables or their complements. 


1 
1 
1 
1 
0 
0 
0 
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A literal is a Boolean variable or its complement. A minterm of the Boolean variables 
X1,Xp,...,X, iS a Boolean product y,y, --- y,, where y; = x; or y; = x;. Hence, a minterm is a 
product of n literals, with one literal for each variable. 


A minterm has the value 1 for one and only one combination of values of its variables. More 
precisely, the minterm y,y,... y, is 1 if and only if each y, is 1, and this occurs if and only 
if x; = 1 when y; = x; and x; = 0 when y; = x;. 


Find a minterm that equals 1 if x, = x; = 0 and x, = x, = x; = 1, and equals O otherwise. 


Solution: The minterm X1x7x3x,4X5 has the correct set of values. < 


By taking Boolean sums of distinct minterms we can build up a Boolean expression with a 
specified set of values. In particular, a Boolean sum of minterms has the value 1 when exactly 
one of the minterms in the sum has the value 1. It has the value 0 for all other combinations of 
values of the variables. Consequently, given a Boolean function, a Boolean sum of minterms 
can be formed that has the value 1 when this Boolean function has the value 1, and has the 
value 0 when the function has the value 0. The minterms in this Boolean sum correspond to 
those combinations of values for which the function has the value 1. The sum of minterms that 
represents the function is called the sum-of-products expansion or the disjunctive normal 
form of the Boolean function. 

(See Exercise 46 in Section 1.3 for the development of disjunctive normal form in propo- 
sitional calculus.) 


Find the sum-of-products expansion for the function F(x, y, z) = (x + y)z. 


Solution: We will find the sum-of-products expansion of F(x, y, z) in two ways. First, we will 
use Boolean identities to expand the product and simplify. We find that 


FQ, y, z) = (@+y)z 
=xXZ+ yz Distributive law 
= xlz + lyz Identity law 
= x(y + y)z + (x +x)yz Unit property 
= xyz + xy Z + xyz + xyz Distributive law 
= xyz + Xy Z + Xy Z. Idempotent law 
Second, we can construct the sum-of-products expansion by determining the values of F for 
all possible values of the variables x, y, and z. These values are found in Table 2. The sum-of- 


products expansion of F is the Boolean sum of three minterms corresponding to the three rows 
of this table that give the value 1 for the function. This gives 


F(x, y, Z) = Xyz + xy Z + Xyz. < 


It is also possible to find a Boolean expression that represents a Boolean function by taking 
a Boolean product of Boolean sums. The resulting expansion is called the conjunctive normal 
form or product-of-sums expansion of the function. These expansions can be found from sum- 
of-products expansions by taking duals. How to find such expansions directly is described in 
Exercise 10. 
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12.2.2 Functional Completeness 


Every Boolean function can be expressed as a Boolean sum of minterms. Each minterm is the 
Boolean product of Boolean variables or their complements. This shows that every Boolean 
function can be represented using the Boolean operators -, +, and ~. Because every Boolean 
function can be represented using these operators we say that the set {-,+,~ } is functionally 
complete. Can we find a smaller set of functionally complete operators? We can do so if one 
of the three operators of this set can be expressed in terms of the other two. This can be done 
using one of De Morgan’s laws. We can eliminate all Boolean sums using the identity 


x+y =X, 


which is obtained by taking complements of both sides in the second De Morgan law, given in 
Table 5 in Section 12.1, and then applying the double complementation law. This means that 
the set {-,~ } is functionally complete. Similarly, we could eliminate all Boolean products using 
the identity 


xy=xt+y, 


which is obtained by taking complements of both sides in the first De Morgan law, given in 
Table 5 in Section 12.1, and then applying the double complementation law. Consequently 
{+,~ } is functionally complete. Note that the set {+,-} is not functionally complete, be- 
cause it is impossible to express the Boolean function F(x) = x using these operators (see 
Exercise 19). 

We have found sets containing two operators that are functionally complete. Can we find 
a smaller set of functionally complete operators, namely, a set containing just one opera- 
tor? Such sets exist. Define two operators, the | or NAND operator, defined by 1 | 1 = 0 and 
1|0=0|1=0]|0=1; and the | or NOR operator, defined by 1 | 1=1)0=0J]1=0Oand 
0 | 0 = 1. Both of the sets { | } and { | } are functionally complete. To see that { | } is function- 
ally complete, because {-,~ } is functionally complete, all that we have to do is show that both 
of the operators - and 7 can be expressed using just the | operator. This can be done as 


aa pee 


=(x|y)| & ly). 


The reader should verify these identities (see Exercise 14). We leave the demonstration that 
{ | } is functionally complete for the reader (see Exercises 15 and 16). 
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Exercises 


. Find a Boolean product of the Boolean variables x, y, 


and z, or their complements, that has the value 1 if and 
only if 

a) x=y=0,z=1. 
ec) x=0y=z=1. 


b) x=0,y=1,z=0. 
d) x=y=z=0. 


. Find the sum-of-products expansions of these Boolean 
functions. 
a) Fwy) =xX+y b) FQ, y) =xy 
c) Fy) =1 d) F(x, y)=y 


. Find the sum-of-products expansions of these Boolean 


functions. 


a) F(x,y, =x+y+z 
b) F(x, y, z) = (x + z)y 
c) F(x, y,z) =x 
d) F(x, y,z)=xy 


. Find the sum-of-products expansions of the Boolean 


function F(x, y, z) that equals 1 if and only if 


a) x=0. b) xy =0. 
c) x+y=0. d) xyz =0. 


. Find the sum-of-products expansion of the Boolean func- 


tion F(w, x, y, z) that has the value 1 if and only if an odd 
number of w, x, y, and z have the value 1. 


. Find the sum-of-products expansion of the Boolean func- 


tion F(x), X2, X3, X4 X5) that has the value 1 if and only if 
three or more of the variables x,, x), x3, X4, and x; have 
the value 1. 


Another way to find a Boolean expression that represents a 
Boolean function is to form a Boolean product of Boolean 
sums of literals. Exercises 7—11 are concerned with represen- 
tations of this kind. 


7. 


Find a Boolean sum containing either x or x, either y 
or y, and either z or z that has the value 0 if and only if 
a) x=y=1,z=0. b) x=y=z=0. 

Q x=z=0,y= 1. 


. Find a Boolean product of Boolean sums of literals that 


has the value 0 if and only if x = y = 1 and z = 0,x =z 

0 and y = 1, or x = y = z = Q. [Hint: Take the Boolean 
product of the Boolean sums found in parts (a), (b), and 
(c) in Exercise 7.] 
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10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


. Show that the Boolean sum y; + yz + + + y,, where y; = 


x; Or y; = X;, has the value 0 for exactly one combina- 
tion of the values of the variables, namely, when x; = 0 if 
y; =x, and x; = 1 if y, = x;. This Boolean sum is called a 
maxterm. 

Show that a Boolean function can be represented as 
a Boolean product of maxterms. This representation is 
called the product-of-sums expansion or conjunctive 
normal form of the function. [Hint: Include one max- 
term in this product for each combination of the variables 
where the function has the value 0.] 

Find the product-of-sums expansion of each of the 
Boolean functions in Exercise 3. 


Express each of these Boolean functions using the oper- 
ators - and 7. 

a) xty+z b) x + yx +z) 

c) x+y d) x(x +y +2) 

Express each of the Boolean functions in Exercise 12 us- 
ing the operators + and `. 
Show that 

a) x=x |x. 

c) xty=(]x) 101). 
Show that 

a) x=xJx. 

b) x=@1xy101»). 
ce) xty=@lyl@ly). 
Show that { | } is functionally complete using Exer- 
cise 15. 


b) xy=(|y)|@|y). 


Express each of the Boolean functions in Exercise 3 us- 
ing the operator | . 


Express each of the Boolean functions in Exercise 3 us- 
ing the operator |. 

Show that the set of operators {+, -} is not functionally 
complete. 


Are these sets of operators functionally complete? 


a{t+,@} bD) o{,@} 


12.3.1 Introduction 


Links » 


Boolean algebra is used to model the circuitry of electronic devices. Each input and each output 
of such a device can be thought of as a member of the set {0, 1}. A computer, or other electronic 


device, is made up of a number of circuits. Each circuit can be designed using the rules of 
Boolean algebra that were studied in Sections 12.1 and 12.2. The basic elements of circuits 
are called gates, and were introduced in Section 1.2. Each type of gate implements a Boolean 
operation. In this section we define several types of gates. Using these gates, we will apply the 


EXAMPLE 1 
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(a) Inverter (b) OR gate (c) AND gate 


FIGURE 1 Basic types of gates. 


rules of Boolean algebra to design circuits that perform a variety of tasks. The circuits that we 
will study in this chapter give output that depends only on the input, and not on the current 
state of the circuit. In other words, these circuits have no memory capabilities. Such circuits are 
called combinational circuits or gating networks. 

We will construct combinational circuits using three types of elements. The first is an 
inverter, which accepts the value of one Boolean variable as input and produces the comple- 
ment of this value as its output. The symbol used for an inverter is shown in Figure 1(a). The 
input to the inverter is shown on the left side entering the element, and the output is shown on 
the right side leaving the element. 

The next type of element we will use is the OR gate. The inputs to this gate are the val- 
ues of two or more Boolean variables. The output is the Boolean sum of their values. The 
symbol used for an OR gate is shown in Figure 1(b). The inputs to the OR gate are shown 
on the left side entering the element, and the output is shown on the right side leaving the 
element. 

The third type of element we will use is the AND gate. The inputs to this gate are the 
values of two or more Boolean variables. The output is the Boolean product of their values. The 
symbol used for an AND gate is shown in Figure 1(c). The inputs to the AND gate are shown 
on the left side entering the element, and the output is shown on the right side leaving the 
element. 

We will permit multiple inputs to AND and OR gates. The inputs to each of these gates are 
shown on the left side entering the element, and the output is shown on the right side. Examples 
of AND and OR gates with n inputs are shown in Figure 2. 


xy xi 
x i 

ea Kiye Xp ae Xp XQ H+ +X, 
Xn : Xn ——> 


FIGURE 2 Gates with n inputs. 


12.3.2 Combinations of Gates 


Combinational circuits can be constructed using a combination of inverters, OR gates, and AND 
gates. When combinations of circuits are formed, some gates may share inputs. This is shown in 
one of two ways in depictions of circuits. One method is to use branchings that indicate all the 
gates that use a given input. The other method is to indicate this input separately for each gate. 
Figure 3 illustrates the two ways of showing gates with the same input values. Note also that 
output from a gate may be used as input by one or more other elements, as shown in Figure 3. 
Both drawings in Figure 3 depict the circuit that produces the output xy + xy. 


Construct circuits that produce the following outputs: (a) (x + y)x, (b) x (y + z), and (c) (x + 
y+z(xyz). 


Solution: Circuits that produce these outputs are shown in Figure 4. < 


860 12/ Boolean Algebra 


= 
kad] 
v y 


FIGURE 3 Two ways to draw the same circuit. 
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FIGURE 4 Circuits that produce the outputs specified in Example 1. 
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12.3.3 Examples of Circuits 


We will give some examples of circuits that perform some useful functions. 


A committee of three individuals decides issues for an organization. Each individual votes either 
yes or no for each proposal that arises. A proposal is passed if it receives at least two yes votes. 
Design a circuit that determines whether a proposal passes. 


Solution: Let x = 1 if the first individual votes yes, and x = 0 if this individual votes no; 
let y = 1 if the second individual votes yes, and y = 0 if this individual votes no; let z = 1 if 
the third individual votes yes, and z = 0 if this individual votes no. Then a circuit must be 
designed that produces the output 1 from the inputs x, y, and z when two or more of x, y, 
and z are 1. One representation of the Boolean function that has these output values is 
xy + xz + yz (see Exercise 12 in Section 12.1). The circuit that implements this function is 
shown in Figure 5. < 


xX XZ 
> Pp xytxetyz 


FIGURE 5 A circuit for majority voting. 


Sometimes light fixtures are controlled by more than one switch. Circuits need to be designed 
so that flipping any one of the switches for the fixture turns the light on when it is off and turns 
the light off when it is on. Design circuits that accomplish this when there are two switches and 
when there are three switches. 


Solution: We will begin by designing the circuit that controls the light fixture when two dif- 
ferent switches are used. Let x = 1 when the first switch is closed and x = 0 when it is open, 
and let y = 1 when the second switch is closed and y = 0 when it is open. Let F(x, y) = 1 when 
the light is on and F(x, y) = 0 when it is off. We can arbitrarily decide that the light will be 
on when both switches are closed, so that F(1, 1) = 1. This determines all the other values 
of F. When one of the two switches is opened, the light goes off, so F(1, 0) = F(O, 1) =0 
When the other switch is also opened, the light goes on, so F(0,0) = 1. Table 1 displays 
these values. Note that F(x, y) = xy + xy. This function is implemented by the circuit shown in 
Figure 6. 
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FIGURE 6 A circuit for a light controlled by two switches. 
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FIGURE7 A circuit for a fixture controlled by three switches. 


We will now design a circuit for three switches. Let x, y, and z be the Boolean variables that 
indicate whether each of the three switches is closed. We let x = | when the first switch is closed, 
and x = 0 when it is open; y = 1 when the second switch is closed, and y = 0 when it is open; 
and z = 1 when the third switch is closed, and z = 0 when it is open. Let F(x, y, z) = 1 when the 
light is on and F(x, y, z) = 0 when the light is off. We can arbitrarily specify that the light be on 
when all three switches are closed, so that F(1, 1, 1) = 1. This determines all other values of F. 
When one switch is opened, the light goes off, so F(1, 1, 0) = FC, 0, 1) = F(O, 1, 1) = 0. When 
a second switch is opened, the light goes on, so F(1, 0, 0) = F(O, 1, 0) = F(0, 0, 1) = 1. Finally, 
when the third switch is opened, the light goes off again, so F(0, 0, 0) = 0. Table 2 shows the 
values of this function. 

The function F can be represented by its sum-of-products expansion as F(x, y, Z) = 
xyz + xy Z + xyz + x yz. The circuit shown in Figure 7 implements this function. 


12.3.4 Adders 


We will illustrate how logic circuits can be used to carry out addition of two positive integers 
from their binary expansions. We will build up the circuitry to do this addition from some 
component circuits. First, we will build a circuit that can be used to find x + y, where x and y 
are two bits. The input to our circuit will be x and y, because these each have the value O or 
the value 1. The output will consist of two bits, namely, s and c, where s is the sum bit and c 
is the carry bit. This circuit is called a multiple output circuit because it has more than one 
output. The circuit that we are designing is called the half adder, because it adds two bits, 
without considering a carry from a previous addition. We show the input and output for the half 
adder in Table 3. From Table 3 we see that c = xy and that s = xy + xy = (x + y)(xy). Hence, 
the circuit shown in Figure 8 computes the sum bit s and the carry bit c from the bits x and y. 
We use the full adder to compute the sum bit and the carry bit when two bits and a carry 
are added. The inputs to the full adder are the bits x and y and the carry c;. The outputs are the 
sum bit s and the new carry c;,,. The inputs and outputs for the full adder are shown in Table 4. 
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y S=Xyc; + Xyc; + XyC; + Xyc; 


Sum = (x + y)(xy) Ci ema | Half > 
WEI) —p| adder 


Ci41 = XC; + XYC;+ 


x — Half xyci + Xyc; 
y > adder xy > 


FIGURE 8 The half adder. FIGURE 9 A full adder. 


vy 


TABLE 4 The two outputs of the full adder, the sum bit s and the carry c;,,, are given by the sum-of- 
Input and products expansions xyc; + xy C; + xyc; +X yc; and xyc; + xyC; + xyc; + xyc,, respectively. How- 
Output for ever, instead of designing the full adder from scratch, we will use half adders to produce the 

the Full Adder. desired output. A full adder circuit using half adders is shown in Figure 9. 
Finally, in Figure 10 we show how full and half adders can be used to add the two three-bit 
integers (xX; Xq). and (y2y1Yo) to produce the sum (53575,59).. Note that s3, the highest-order 


bit in the sum, is given by the carry c3. 


1y;1i};1]1 1 
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FIGURE 10 Adding two three-bit 
integers with full and half adders. 
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= 


me 


6. Construct circuits from inverters, AND gates, and OR 
gates to produce these outputs. 

b) (w+ y)x 

d) ®+20+2) 


7. Design a circuit that implements majority voting for five 
individuals. 


a) x+y 
c) xyzt+xyz 


8. Design a circuit for a light fixture controlled by four 
switches, where flipping one of the switches turns the 
light on when it is off and turns it off when it is on. 

9. Show how the sum of two five-bit integers can be found 
using full and half adders. 

10. Construct a circuit for a half subtractor using AND gates, 
OR gates, and inverters. A half subtractor has two bits 
as input and produces as output a difference bit and a bor- 
row. 

11. Construct a circuit for a full subtractor using AND gates, 
OR gates, and inverters. A full subtractor has two bits 
and a borrow as input, and produces as output a difference 
bit and a borrow. 

12. Use the circuits from Exercises 10 and 11 to find the dif- 
ference of two four-bit integers, where the first integer is 
greater than the second integer. 

* 13. Construct a circuit that compares the two-bit integers 
(xxo) and (y; Yo), returning an output of 1 when the first 
of these numbers is larger and an output of 0 otherwise. 

*14. Construct a circuit that computes the product of the two- 
bit integers (x,xX9). and (y, Yo). The circuit should have 
four output bits for the bits in the product. 


Minimization of Circuits 


Two gates that are often used in circuits are NAND and NOR 
gates. When NAND or NOR gates are used to represent cir- 
cuits, no other types of gates are needed. The notation for 
these gates is as follows: 


3 x NAND y P x NOR y 
y—> y 


*15. Use NAND gates to construct circuits with these out- 


puts. 
a) x b) x+y 
c) xy d) x®y 


* 16. Use NOR gates to construct circuits for the outputs given 
in Exercise 15. 


*17. Construct a half adder using NAND gates. 


*18. Construct a half adder using NOR gates. 


A multiplexer is a switching circuit that produces as output 
one of a set of input bits based on the value of control bits. 


19. Construct a multiplexer using AND gates, OR gates, and 
inverters that has as input the four bits x9, x,, X2, and x, 
and the two control bits cg and c,. Set up the circuit so 
that x; is the output, where i is the value of the two-bit 
integer (C,Cp)>. 


The depth of a combinatorial circuit can be defined by speci- 
fying that the depth of the initial input is 0 and if a gate has n 
different inputs at depths d,, d), ..., d,,, respectively, then its 
outputs have depth equal to max(d,, d>, ..., d,,) + 1; this value 
is also defined to be the depth of the gate. The depth of a com- 
binatorial circuit is the maximum depth of the gates in the 


circuit. 
20. Find the depth of 
a) the circuit constructed in Example 2 for majority vot- 
ing among three people. 
b) the circuit constructed in Example 3 for a light con- 
trolled by two switches. 
c) the half adder shown in Figure 8. 
d) the full adder shown in Figure 9. 


12.4.1 Introduction 


The efficiency of a combinational circuit depends on the number and arrangement of its gates. 
The process of designing a combinational circuit begins with the table specifying the output for 
each combination of input values. We can always use the sum-of-products expansion of a circuit 
to find a set of logic gates that will implement this circuit. However, the sum-of-products expan- 
sion may contain many more terms than are necessary. Terms in a sum-of-products expansion 
that differ in just one variable, so that in one term this variable occurs and in the other term the 
complement of this variable occurs, can be combined. For instance, consider the circuit that has 
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FIGURE 1 Two circuits with the same output. 


output 1 if and only if x = y = z = 1 or x = z = 1 and y = 0. The sum-of-products expansion 
of this circuit is xyz + xyz. The two products in this expansion differ in exactly one variable, 
namely, y. They can be combined as 


xyz + xyz = (y + y)(xz) 
= 1- (xz) 


= XZ. 


Hence, xz is a Boolean expression with fewer operators that represents the circuit. We show 
two different implementations of this circuit in Figure 1. The second circuit uses only one gate, 
whereas the first circuit uses three gates and an inverter. 

This example shows that combining terms in the sum-of-products expansion of a circuit 
leads to a simpler expression for the circuit. We will describe two procedures that simplify 
sum-of-products expansions. 

The goal of both procedures is to produce Boolean sums of Boolean products that represent 
a Boolean function with the fewest products of literals such that these products contain the 
fewest literals possible among all sums of products that represent a Boolean function. Finding 
such a sum of products is called minimization of the Boolean function. Minimizing a Boolean 
function makes it possible to construct a circuit for this function that uses the fewest gates and 
fewest inputs to the AND gates and OR gates in the circuit, among all circuits for the Boolean 
expression we are minimizing. 

Until the early 1960s logic gates were individual components. To reduce costs it was impor- 
tant to use the fewest gates to produce a desired output. However, in the mid-1960s, integrated 
circuit technology was developed that made it possible to combine gates on a single chip. Even 
though it is now possible to build increasingly complex integrated circuits on chips at low cost, 
minimization of Boolean functions remains important. 

Reducing the number of gates on a chip can lead to a more reliable circuit and can reduce 
the cost to produce the chip. Also, minimization makes it possible to fit more circuits on the 
same chip. Furthermore, minimization reduces the number of inputs to gates in a circuit. This 
reduces the time used by a circuit to compute its output. Moreover, the number of inputs to a 
gate may be limited because of the particular technology used to build logic gates. 

The first procedure we will introduce, known as Karnaugh maps (or K-maps), was designed 
in the 1950s to help minimize circuits by hand. K-maps are useful in minimizing circuits with 
up to six variables, although they become rather complex even for five or six variables. The 
second procedure we will describe, the Quine—McCluskey method, was invented in the 1960s. 
It automates the process of minimizing combinatorial circuits and can be implemented as a 
computer program. 
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FIGURE 2 
K-maps in two 
variables. 


EXAMPLE 1 


Courtesy of Maurice 
Karnaugh 


COMPLEXITY OF BOOLEAN FUNCTION MINIMIZATION Unfortunately, minimizing 
Boolean functions with many variables is a computationally intensive problem. It has been 
shown that this problem is an NP-complete problem (see Section 3.3 and [Ka93]), so the exis- 
tence of a polynomial-time algorithm for minimizing Boolean circuits is unlikely. The Quine- 
McCluskey method has exponential complexity. In practice, it can be used only when the num- 
ber of literals does not exceed ten. Since the 1970s a number of newer algorithms have been 
developed for minimizing combinatorial circuits (see [Ha93] and [KaBe04]). However, with 
the best algorithms yet devised, only circuits with no more than 25 variables can be minimized. 
Also, heuristic (or rule-of-thumb) methods can be used to substantially simplify, but not neces- 
sarily minimize, Boolean expressions with a larger number of literals. 


12.4.2 Karnaugh Maps 


To reduce the number of terms in a Boolean expression representing a circuit, it is necessary 
to find terms to combine. There is a graphical method, called a Karnaugh map or K-map, for 
finding terms to combine for Boolean functions involving a relatively small number of variables. 
The method we will describe was introduced by Maurice Karnaugh in 1953. His method is based 
on earlier work by E. W. Veitch. (This method is usually applied only when the function involves 
six or fewer variables.) K-maps give us a visual method for simplifying sum-of-products expan- 
sions; they are not suited for mechanizing this process. We will first illustrate how K-maps are 
used to simplify expansions of Boolean functions in two variables. We will continue by show- 
ing how K-maps can be used to minimize Boolean functions in three variables and then in four 
variables. Then we will describe the concepts that can be used to extend K-maps to minimize 
Boolean functions in more than four variables. 

There are four possible minterms in the sum-of-products expansion of a Boolean function 
in the two variables x and y. A K-map for a Boolean function in these two variables consists 
of four cells, where a 1 is placed in the cell representing a minterm if this minterm is present 
in the expansion. Cells are said to be adjacent if the minterms that they represent differ in 
exactly one literal. For instance, the cell representing xy is adjacent to the cells representing xy 
and xy. The four cells and the terms that they represent are shown in Figure 2. 


Find the K-maps for (a) xy + xy, (b) xy + xy, and (c) xy + xy + xy. 


Solution: We include a 1 in a cell when the minterm represented by this cell is present in the 
sum-of-products expansion. The three K-maps are shown in Figure 3. < 


We can identify minterms that can be combined from the K-map. Whenever there are 1s in 
two adjacent cells in the K-map, the minterms represented by these cells can be combined into a 
product involving just one of the variables. For instance, xy and xy are represented by adjacent 
cells and can be combined into y, because xy + xy = (x + x)y = y. Moreover, if 1s are in all four 
cells, the four minterms can be combined into one term, namely, the Boolean expression | that 
involves none of the variables. We circle blocks of cells in the K-map that represent minterms 
that can be combined and then find the corresponding sum of products. The goal is to identify 


MAURICE KARNAUGH (BORN 1924) Maurice Karnaugh, born in New York City, received his B.S. from 
the City College of New York and his M.S. and Ph.D. from Yale University. He was a member of the technical 
staff at Bell Laboratories from 1952 until 1966 and Manager of Research and Development at the Federal 
Systems Division of AT&T from 1966 to 1970. In 1970 he joined IBM as a member of the research staff. 
Karnaugh has made fundamental contributions to the application of digital techniques in both computing and 
telecommunications. His current interests include knowledge-based systems in computers and heuristic search 
methods. 
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FIGURE3 K-maps for the sum-of-products expansions in Example 1. 


the largest possible blocks, and to cover all the 1s with the fewest blocks using the largest blocks 
first and always using the largest possible blocks. 


Simplify the sum-of-products expansions given in Example 1. 


Solution: The grouping of minterms is shown in Figure 4 using the K-maps for these expansions. 
Minimal expansions for these sums-of-products are (a) y, (b) xy + xy, and (c) x + y. < 


© 


(a) (b) (c) 


FIGURE 4 Simplifying the sum-of-products expansions from Example 2. 


A K-map in three variables is a rectangle divided into eight cells. The cells represent the 
eight possible minterms in three variables. Two cells are said to be adjacent if the minterms that 
they represent differ in exactly one literal. One of the ways to form a K-map in three variables 
is shown in Figure 5(a). This K-map can be thought of as lying on a cylinder, as shown in 
Figure 5(b). On the cylinder, two cells have a common border if and only if they are adjacent. 


yz yz yz yz 
$ xyz XYZ XYZ xyz Vi <1 
XyZ | _ xyz 
on a BD 
x Xyz Xyz xyz Xyz XYZ—— i| «Iyá 
XYZ XYZ 
Pr cae (emia $ 
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FIGURE 5 K-maps in three variables. 
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FIGURE 6 Blocks in K-maps in three variables. 


EXAMPLE 3 


To simplify a sum-of-products expansion in three variables, we use the K-map to iden- 
tify blocks of minterms that can be combined. Blocks of two adjacent cells represent pairs of 
minterms that can be combined into a product of two literals; 2 x 2 and 4 x 1 blocks of cells 
represent minterms that can be combined into a single literal; and the block of all eight cells 
represents a product of no literals, namely, the function 1. In Figure 6, 1 x 2,2 x 1,2 *2,4x 1, 
and 4 x 2 blocks and the products they represent are shown. 

The product of literals corresponding to a block of all 1s in the K-map is called an implicant 
of the function being minimized. It is called a prime implicant if this block of 1s is not contained 
in a larger block of 1s representing the product of fewer literals than in this product. 

The goal is to identify the largest possible blocks in the map and cover all the 1s in the map 
with the least number of blocks, using the largest blocks first. The largest possible blocks are 
always chosen, but we must always choose a block if it is the only block of 1s covering a 1 in 
the K-map. Such a block represents an essential prime implicant. By covering all the 1s in the 
map with blocks corresponding to prime implicants we can express the sum of products as a 
sum of prime implicants. Note that there may be more than one way to cover all the 1s using 
the least number of blocks. 

Example 3 illustrates how K-maps in three variables are used. 


Use K-maps to minimize these sum-of-products expansions. 


(a) xyz + xyz + xyz + xyz 

(b) xyz +xyZ+Xyzt+xXVZ+XVZ 

(c) xyz + xyz+ xyz + Xy Z + Xyz +XYZ + XYZ 
(d) xyz + xyz + xyz +xyz 


Solution: The K-maps for these sum-of-products expansions are shown in Figure 7. The group- 
ing of blocks shows that minimal expansions into Boolean sums of Boolean products are 
(a) xz + yz + xyz, (b) y + Xz, (c) x+y +z, and (d) xz + xy. In part (d) note that the prime 
implicants xz and xy are essential prime implicants, but the prime implicant yz is a 
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FIGURE 7 Using K-maps in three variables. 


prime implicant that is not essential, because the cells it covers are covered by the other two 
prime implicants. < 


A K-map in four variables is a square that is divided into 16 cells. The cells represent the 16 
possible minterms in four variables. One of the ways to form a K-map in four variables is shown 
in Figure 8. 

Two cells are adjacent if and only if the minterms they represent differ in one literal. Con- 
sequently, each cell is adjacent to four other cells. The K-map of a sum-of-products expansion 
in four variables can be thought of as lying on a torus, so that adjacent cells have a common 
boundary (see Exercise 28). The simplification of a sum-of-products expansion in four vari- 
ables is carried out by identifying those blocks of 2, 4, 8, or 16 cells that represent minterms 
that can be combined. Each cell representing a minterm must either be used to form a product 
using fewer literals, or be included in the expansion. In Figure 9 some examples of blocks that 
represent products of three literals, products of two literals, and a single literal are illustrated. 

As is the case in K-maps in two and three variables, the goal is to identify the largest 
blocks of 1s in the map that correspond to the prime implicants and to cover all the 1s using the 
fewest blocks needed, using the largest blocks first. The largest possible blocks are always used. 
Example 4 illustrates how K-maps in four variables are used. 


yz yz yz yz 


wx] WXYZ | WXYZ | wxyz | wxyz 


wX| wXyz | WXyZ | wxyz | wxyz 


Wx | wxyz | Wwxyz | wxyz | wxyz 


wx | wxyz WxyZ | WXYZ WXYZ 


FIGURE 8 K-maps in four variables. 
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EXAMPLE 4 


XZ = wxyz + wxyz + 
wxyz + WXYZ 


Z = WXYZ + WXYZ + wxyz + 
WXYZ + WXYZ + WXYZ + WXYZ + WXYZ 


(c) (d) 


FIGURE 9 Blocks in K-maps in four variables. 


Use K-maps to simplify these sum-of-products expansions. 


(a) wxyz + wxyz + wxyZ + wxyz + WXYZ + WXYZ + Wxyz + 
W XYZ + W XYZ 

(b) wxyZ + wxyz + wxyzt+ WXYZ + WXYZ + WXYZ +WXyZ 

(c) wxyz + wxy z + wxyz + wxyzZ + WX yZ + Wxyz + WxyZ + WxyZ+ 
wxyz +w xyz + wxyz 


Solution: The K-maps for these expansions are shown in Figure 10. Using the blocks shown 
leads to the sum of products (a) wyz + wxz + wx y + w xy + wxyz, (b) yz + wxy + Xz, and (c) z + 
wx + wxy. The reader should determine whether there are other choices of blocks in each part 
that lead to different sums of products representing these Boolean functions. < 


K-maps can realistically be used to minimize Boolean functions with five or six variables, 
but beyond that, they are rarely used because they become extremely complicated. However, 
the concepts used in K-maps play an important role in newer algorithms. Furthermore, master- 
ing these concepts helps you understand these newer algorithms and the computer-aided design 
(CAD) programs that implement them. As we develop these concepts, we will be able to illus- 
trate them by referring back to our discussion of minimization of Boolean functions in three 
and in four variables. 

The K-maps we used to minimize Boolean functions in two, three, and four variables are 
built using 2 x 2, 2 x 4, and 4 x 4 rectangles, respectively. Furthermore, corresponding cells in 
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FIGURE 10 Using K-maps in four variables. 


the top row and bottom row and in the leftmost column and rightmost column in each of these 
cases are considered adjacent because they represent minterms differing in only one literal. We 
can build K-maps for minimizing Boolean functions in more than four variables in a similar 
way. We use a rectangle containing 2!"/7! rows and 2!"/2! columns. (These K-maps contain 
2” cells because [1/2] + |n/2] = n.) The rows and columns need to be positioned so that the 
cells representing minterms differing in just one literal are adjacent or are considered adjacent 
by specifying additional adjacencies of rows and columns. To help (but not entirely) achieve 
this, the rows and columns of a K-map are arranged using Gray codes (see Section 10.5), where 
we associate bit strings and products by specifying that a 1 corresponds to the appearance of 
a variable and a O with the appearance of its complement. For example, in a 10-dimensional 
K-map, the Gray code 01110 used to label a row corresponds to the product x, xX3X3X4X5. 


The K-maps we used to minimize Boolean functions with four variables have four rows and 
four columns. Both the rows and the columns are arranged using the Gray code 11,10,00,01. 
The rows represent products wx, wx, wx, and wx, respectively, and the columns correspond 
to the products yz, yz, y z, and yz, respectively. Using Gray codes and considering cells adjacent 
in the first and last rows and in the first and last columns, we ensured that minterms that differ 
in only one variable are always adjacent. < 


To minimize Boolean functions in five variables we use K-maps with 2* = 8 columns and 
2? = 4 rows. We label the four rows using the Gray code 11,10,00,01, corresponding to 
XX2, X1X2, X,Xp, and xx, respectively. We label the eight columns using the Gray code 
111,110,100,101,001,000,010,011 corresponding to the terms x3X4X5, X3X4X5, X3X4X5, X3X4X5, 
X3X 4X5, X3X4X5, XzX4X5, aNd x3x4X5, respectively. Using Gray codes to label columns and rows 
ensures that the minterms represented by adjacent cells differ in only one variable. However, 
to make sure all cells representing products that differ in only one variable are considered ad- 
jacent, we consider cells in the top and bottom rows to be adjacent, as well as cells in the first 
and eighth columns, the first and fourth columns, the second and seventh columns, the third and 
sixth columns, and the fifth and eighth columns (as the reader should verify). < 


To use a K-map to minimize a Boolean function in n variables, we first draw a K-map of 
the appropriate size. We place 1s in all cells corresponding to minterms in the sum-of-products 
expansion of this function. We then identify all prime implicants of the Boolean function. To do 
this we look for the blocks consisting of 2* clustered cells all containing a 1, where 1 < k < n. 
These blocks correspond to the product of n — k literals. (Exercise 33 asks the reader to verify 
this.) Furthermore, a block of 2* cells each containing a 1 not contained in a block of 2**! 
cells each containing a | represents a prime implicant. The reason that this implicant is a prime 
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EXAMPLE 7 


Links » 


EXAMPLE 8 


implicant is that no product obtained by deleting a literal is also represented by a block of cells 
all containing 1s. 


A block of eight cells representing a product of two literals in a K-map for minimizing Boolean 
functions in five variables all containing 1s is a prime implicant if it is not contained in a larger 
block of 16 cells all containing 1s representing a single literal. < 


Once all prime implicants have been identified, the goal is to find the smallest possible sub- 
set of these prime implicants with the property that the cells representing these prime implicants 
cover all the cells containing a | in the K-map. We begin by selecting the essential prime im- 
plicants because each of these is represented by a block that covers a cell containing a 1 that is 
not covered by any other prime implicant. We add additional prime implicants to ensure that all 
1s in the K-map are covered. When the number of variables is large, this last step can become 
exceedingly complicated. 


12.4.3 Don’t Care Conditions 


In some circuits we care only about the output for some combinations of input values, because 
other combinations of input values are not possible or never occur. This gives us freedom in 
producing a simple circuit with the desired output because the output values for all those com- 
binations that never occur can be arbitrarily chosen. The values of the function for these combi- 
nations are called don’t care conditions. A d is used in a K-map to mark those combinations of 
values of the variables for which the function can be arbitrarily assigned. In the minimization 
process we can assign Is as values to those combinations of the input values that lead to the 
largest blocks in the K-map. This is illustrated in Example 8. 


One way to code decimal expansions using bits is to use the four bits of the binary expansion 
of each digit in the decimal expansion. For instance, 873 is encoded as 100001110011. This 
encoding of a decimal expansion is called a binary coded decimal expansion. Because there 
are 16 blocks of four bits and only 10 decimal digits, there are six combinations of four bits that 
are not used to encode digits. Suppose that a circuit is to be built that produces an output of 1 if 
the decimal digit is 5 or greater and an output of 0 if the decimal digit is less than 5. How can 
this circuit be simply built using OR gates, AND gates, and inverters? 


Solution: Let F(w, x, y, z) denote the output of the circuit, where wxyz is a binary expansion 
of a decimal digit. The values of F are shown in Table 1. The K-map for F, with ds in the 
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FIGURE 11 The K-map for F showing its don’t care positions. 


don’t care positions, is shown in Figure 11(a). We can either include or exclude squares with 
ds from blocks. This gives us many possible choices for the blocks. For example, excluding 
all squares with ds and forming blocks, as shown in Figure 11(b), produces the expression 
wx y + wxy + wxz. Including some of the ds and excluding others and forming blocks, as shown 
in Figure 11(c), produces the expression wx + wxy + xyz. Finally, including all the ds and using 
the blocks shown in Figure 11(d) produces the simplest sum-of-products expansion possible, 
namely, F(x, y, z) = w + xy + xz. < 


12.4.4 The Quine—McCluskey Method 


We have seen that K-maps can be used to produce minimal expansions of Boolean functions 
as Boolean sums of Boolean products. However, K-maps are awkward to use when there are 


EDWARD J. McCLUSKEY (1929-2016) Edward McCluskey attended Bowdoin College and M.I.T., where 
he received his doctorate in electrical engineering in 1956. He joined Bell Telephone Laboratories in 1955, 
remaining there until 1959. McCluskey was professor of electrical engineering at Princeton University from 
1959 until 1966, also serving as director of the Computer Center at Princeton from 1961 to 1966. In 1967 he 
took a position as professor of computer science and electrical engineering at Stanford University, where he 
also served as director of the Digital Systems Laboratory from 1969 to 1978. McCluskey worked in a variety of 


| areas in computer science, including fault-tolerant computing, computer architecture, testing, and logic design. 


He was the director of the Center for Reliable Computing at Stanford University and an ACM Fellow. 
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more than four variables. Furthermore, the use of K-maps relies on visual inspection to identify 
terms to group. For these reasons there is a need for a procedure for simplifying sum-of-products 
expansions that can be mechanized. The Quine—McCluskey method is such a procedure. It can 
be used for Boolean functions in any number of variables. It was developed in the 1950s by 
W. V. Quine and E. J. McCluskey, Jr. Basically, the Quine—McCluskey method consists of two 
parts. The first part finds those terms that are candidates for inclusion in a minimal expansion 
as a Boolean sum of Boolean products. The second part determines which of these terms to 
actually use. We will use Example 9 to illustrate how, by successively combining implicants 
into implicants with one fewer literal, this procedure works. 


EXAMPLE 9 We will show how the Quine—McCluskey method can be used to find a minimal expansion 
equivalent to 


xyz + xyz + xyz + xyz+xyz. 


We will represent the minterms in this expansion by bit strings. The first bit will be 1 if x 
occurs and 0 if x occurs. The second bit will be 1 if y occurs and 0 if y occurs. The third bit will 
be 1 if z occurs and 0 if z occurs. We then group these terms according to the number of 1s in 
the corresponding bit strings. This information is shown in Table 2. 

Minterms that can be combined are those that differ in exactly one literal. Hence, two terms 
that can be combined differ by exactly one in the number of 1s in the bit strings that represent 
them. When two minterms are combined into a product, this product contains two literals. A 
product in two literals is represented using a dash to denote the variable that does not occur. For 
instance, the minterms xyz and x yz, represented by bit strings 101 and 001, can be combined into 
yz, represented by the string —01. All pairs of minterms that can be combined and the product 
formed from these combinations are shown in Table 3. 

Next, all pairs of products of two literals that can be combined are combined into one lit- 
eral. Two such products can be combined if they contain literals for the same two variables, 
and literals for only one of the two variables differ. In terms of the strings representing the 
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products, these strings must have a dash in the same position and must differ in exactly one 
of the other two slots. We can combine the products yz and yz, represented by the strings —1 1 
and —01, into z, represented by the string -—1. We show all the combinations of terms that can 
be formed in this way in Table 3. 

In Table 3 we also indicate which terms have been used to form products with fewer literals; 
these terms will not be needed in a minimal expansion. The next step is to identify a minimal 
set of products needed to represent the Boolean function. We begin with all those products that 
were not used to construct products with fewer literals. Next, we form Table 4, which has a row 
for each candidate product formed by combining original terms, and a column for each original 
term; and we put an X in a position if the original term in the sum-of-products expansion was 
used to form this candidate product. In this case, we say that the candidate product covers 
the original minterm. We need to include at least one product that covers each of the original 
minterms. Consequently, whenever there is only one X in a column in the table, the product 
corresponding to the row this X is in must be used. From Table 4 we see that both z and xy are 
needed. Hence, the final answer is z+ xy. < 


As was illustrated in Example 9, the Quine—-McCluskey method uses this sequence of steps 
to simplify a sum-of-products expression. 


1. Express each minterm in n variables by a bit string of length n with a 1 in the ith position 
if x; occurs and a 0 in this position if x; occurs. 


2. Group the bit strings according to the number of 1s in them. 


3. Determine all products in n — 1 variables that can be formed by taking the Boolean sum 
of minterms in the expansion. Minterms that can be combined are represented by bit 
strings that differ in exactly one position. Represent these products in n — 1 variables 
with strings that have a | in the ith position if x; occurs in the product, a 0 in this position 
if x; occurs, and a dash in this position if there is no literal involving x; in the product. 


| WILLARD VAN ORMAN QUINE (1908-2000) Willard Quine, born in Akron, Ohio, attended Oberlin Col- 
| lege and later Harvard University, where he received his Ph.D. in philosophy in 1932. He became a Junior 


Fellow at Harvard in 1933 and was appointed to a position on the faculty there in 1936. He remained at Harvard 
his entire professional life, except for World War II, when he worked for the U.S. Navy decrypting messages 
from German submarines. Quine was always interested in algorithms, but not in hardware. He arrived at his 
discovery of what is now called the Quine—McCluskey method as a device for teaching mathematical logic, 
rather than as a method for simplifying switching circuits. Quine was one of the most famous philosophers 
of the twentieth century. He made fundamental contributions to the theory of knowledge, mathematical logic 
and set theory, and the philosophies of logic and language. His books, including New Foundations of Mathe- 
matical Logic published in 1937 and Word and Object published in 1960, have had a profound impact. Quine 


retired from Harvard in 1978 but continued to commute from his home in Beacon Hill to his office there. He used the 1927 Remington 
typewriter on which he prepared his doctoral thesis for his entire life. He even had an operation performed on this machine to add 
a few special symbols, removing the second period, the second comma, and the question mark. When asked whether he missed the 
question mark, he replied, “Well, you see, I deal in certainties.” There is even a word quine, defined in the New Hacker’s Dictionary 
as a program that generates a copy of its own source code as its complete output. Producing the shortest possible quine in a given 
programming language is a popular puzzle for hackers. 
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EXAMPLE 10 


4. Determine all products in n — 2 variables that can be formed by taking the Boolean sum 
of the products in n — 1 variables found in the previous step. Products in n — 1 variables 
that can be combined are represented by bit strings that have a dash in the same position 
and differ in exactly one position. 

5. Continue combining Boolean products into products in fewer variables as long as 
possible. 

6. Find all the Boolean products that arose that were not used to form a Boolean product in 
one fewer literal. 

7. Find the smallest set of these Boolean products such that the sum of these products rep- 
resents the Boolean function. This is done by forming a table showing which minterms 
are covered by which products. Every minterm must be covered by at least one product. 
The first step in using this table is to find all essential prime implicants. Each essential 
prime implicant must be included because it is the only prime implicant that covers one 
of the minterms. Once we have found essential prime implicants, we can simplify the 
table by eliminating the rows for minterms covered by these prime implicants. Further- 
more, we can eliminate any prime implicants that cover a subset of minterms covered by 
another prime implicant (as the reader should verify). Moreover, we can eliminate from 
the table the row for a minterm if there is another minterm that is covered by a subset of 
the prime implicants that cover this minterm. This process of identifying essential prime 
implicants that must be included, followed by eliminating redundant prime implicants 
and identifying minterms that can be ignored, is iterated until the table does not change. 
At this point we use a backtracking procedure to find the optimal solution where we add 
prime implicants to the cover to find possible solutions, which we compare to the best 
solution found so far at each step. 


A final example will illustrate how this procedure is used to simplify a sum-of-products 
expansion in four variables. 


Use the Quine—McCluskey method to simplify the sum-of-products expansion wxyz + wxyz + 
WXYZ + WXYZ + WXYZ + w xyz + WX yz. 


Solution: We first represent the minterms by bit strings and then group these terms together 
according to the number of 1s in the bit strings. This is shown in Table 5. All the Boolean 
products that can be formed by taking Boolean sums of these products are shown in Table 6. 
The only products that were not used to form products in fewer variables are wz, wyz, wxy, 
and xyz. In Table 7 we show the minterms covered by each of these products. To cover these 
minterms we must include wz and wyz, because these products are the only products that cover 
wxyz and wxyz, respectively. Once these two products are included, we see that only one of the 
two products left is needed. Consequently, we can take either wz + wyz + wxy or wz + wyz + xyz 
as the final answer. < 


TABLE 5 
Bit Sring 
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TABLE 6 
es a 


(3,5,6,7) 


TABLE 7 


Exercises 
1. a) Draw a K-map for a function in two variables and put 5. a) Draw a K-map for a function in three variables. Put a 
a 1 in the cell representing xy. 1 in the cell that represents xyz. 
b) What are the minterms represented by cells adjacent b) Which oe ms are represented by cells adjacent to 
to this cell? Hosea 
6. Use K-maps to find simpler circuits with the same output 
2. Find the sum-of-products expansions represented by each as each of the circuits shown. 
of these K-maps. een 
a) me xyz + Xyz 
a v J W a D v J y > 7ye 


3. Draw the K-maps of these sum-of-products expansions b) x XYZ 
in two variables. y Z > 


a) xy b) xy+xy xyz + xyz + Xyz + xyz 
c) xyt+xytxytxy A EA q >= ‘ 
4. Use a K-map to find a minimal expansion as a Boolean ee S =: > 
sum of Boolean products of each of these functions of the x—Do+ He 
Boolean variables x and y. y —— - 
a) xy +37 i i 
_ Pa ee io 
b) iye ES i Ror 
c) xy +xy + Xy +XYy z—Bo pl J 
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10. 


11. 


12. 


13. 


14. 
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c) a £ 7 2 ER" 
i oes D yz aye t+z)+ 9+] 
>) 
> 


Rae: Se 


Xz. 
ely, 
y—po> F+zZ 
z—po4 Gt yO +z) 


. Draw the K-maps of these sum-of-products expansions 


in three variables. 
a) xyz 
c) xyz+xyz + Xyz + xyz 


b) xyz + xyz 


. Construct a K-map for F(x, y, z) = xz + yz + xyz. Use this 


K-map to find the implicants, prime implicants, and es- 
sential prime implicants of F(x, y, z). 


. Construct a K-map for F(x, y, z) = xz + xyz + yz. Use this 


K-map to find the implicants, prime implicants, and es- 
sential prime implicants of F(x, y, z). 


Draw the 3-cube Q, and label each vertex with the 
minterm in the Boolean variables x, y, and z associated 
with the bit string represented by this vertex. For each 
literal in these variables indicate the 2-cube Q, that is a 
subgraph of Q, and represents this literal. 


Draw the 4-cube Q, and label each vertex with the 
minterm in the Boolean variables w, x, y, and z associ- 
ated with the bit string represented by this vertex. For 
each literal in these variables, indicate which 3-cube Q, 
that is a subgraph of Q, represents this literal. Indicate 
which 2-cube Q, that is a subgraph of Q, represents the 
products wz, xy, and yz. 


Use a K-map to find a minimal expansion as a Boolean 
sum of Boolean products of each of these functions in the 
variables x, y, and z. 


a) xyz + xyz 
b) xyz + xyz + xyz + xyz 


C) Xyz + xyz + xy Z + Xyz + xyz 
d) xyz + xyz + xy Z + Xyz + Xyz + Xyz 


a) Draw a K-map for a function in four variables. Put a 
1 in the cell that represents wxyz. 


b) Which minterms are represented by cells adjacent to 
this cell? 


Use a K-map to find a minimal expansion as a Boolean 
sum of Boolean products of each of these functions in the 
variables w, x, y, and z. 
a) wxyz + wxyz + wxyZ + WXyZ + wx yz 
b) wxyz + wxyz + wxyz + wxyz + Wxyz + wx yz 
€) wxyz + wxyz + wxyz + wx yz + wxyzt 
Wxyz + WXYZ + WX yz 
d) wxyz + wxyz + wxyz + wxyz + wxyz + 
wxyz + wxyz + wxyz + wx yz 


15. 


16. 


17. 


18. 


19. 


* 20. 


*21. 


22. 


23. 


24. 


25. 


*26. 


27. 


* 28. 


Find the cells in a K-map for Boolean functions with five 
variables that correspond to each of these products. 


e) XX4 
e) x3 f) X; 
How many cells in a K-map for Boolean functions 


with six variables are needed to represent x), XX6, 
X1 XX6; X2X3X4X5, and xı X,xX4X5, respectively? 


a) X1X2X3X4 b) X1X3x5 


d) zz 


a) How many cells does a K-map in six variables have? 


b) How many cells are adjacent to a given cell in a K- 
map in six variables? 


Show that cells in a K-map for Boolean functions in five 
variables represent minterms that differ in exactly one 
literal if and only if they are adjacent or are in cells 
that become adjacent when the top and bottom rows and 
cells in the first and eighth columns, the first and fourth 
columns, the second and seventh columns, the third and 
sixth columns, and the fifth and eighth columns are con- 
sidered adjacent. 


Which rows and which columns of a 4x 16 map for 
Boolean functions in six variables using the Gray codes 
1111, 1110, 1010, 1011, 1001, 1000, 0000, 0001, 0011, 
0010, 0110, 0111, 0101, 0100, 1100, 1101 to label the 
columns and 11, 10, 00, 01 to label the rows need to be 
considered adjacent so that cells that represent minterms 
that differ in exactly one literal are considered adjacent? 


Use K-maps to find a minimal expansion as a Boolean 
sum of Boolean products of Boolean functions that have 
as input the binary code for each decimal digit and pro- 
duce as output a 1 if and only if the digit corresponding 
to the input is 

a) odd. 

c) not 4,5, or 6. 
Suppose that there are five members on a committee, 
but that Smith and Jones always vote the opposite of 
Marcus. Design a circuit that implements majority voting 
of the committee using this relationship between votes. 


b) not divisible by 3. 


Use the Quine-McCluskey method to simplify the sum- 
of-products expansions in Example 3. 


Use the Quine—McCluskey method to simplify the sum- 
of-products expansions in Exercise 12. 


Use the Quine—McCluskey method to simplify the sum- 
of-products expansions in Example 4. 


Use the Quine—McCluskey method to simplify the sum- 
of-products expansions in Exercise 14. 


Explain how K-maps can be used to simplify product-of- 
sums expansions in three variables. [Hint: Mark with a0 
all the maxterms in an expansion and combine blocks of 
maxterms. | 


Use the method from Exercise 26 to simplify the product- 
of-sums expansion (x+y+z)\(x+y+Za+y+zZ) 
(x+y+z(x+y+z). 

Draw a K-map for the 16 minterms in four Boolean vari- 
ables on the surface of a torus. 


29. Build a circuit using OR gates, AND gates, and inverters 
that produces an output of 1 if a decimal digit, encoded 
using a binary coded decimal expansion, is divisible by 
3, and an output of 0 otherwise. 

In Exercises 30-32 find a minimal sum-of-products expan- 

sion, given the K-map shown with don’t care conditions indi- 

cated with ds. 


30. yz yz YZ Vz 31. yz yk JZ Vz 
wx} d|1]d|1 wx| 1 1 
wx d|d wx djl 
wX d wX lid 
wx td wx| d d 

Key Terms and Results 

TERMS 


Boolean variable: a variable that assumes only the values 0 
and 1 


x (complement of x): an expression with the value 1 when x 
has the value 0 and the value 0 when x has the value 1 


x -y (or xy) (Boolean product or conjunction of x and y): 
an expression with the value 1 when both x and y have the 
value | and the value 0 otherwise 


x + y (Boolean sum or disjunction of x and y): an expression 
with the value 1 when either x or y, or both, has the value 
1, and 0 otherwise 


Boolean expressions: the expressions obtained recursively by 
specifying that 0, 1, x,,..., x,, are Boolean expressions and 
E,, (E, + Ey), and (E, E,) are Boolean expressions if E; and 
E, are 

dual of a Boolean expression: the expression obtained by in- 


terchanging + signs and - signs and interchanging Os and 
Is 


Boolean function of degree n: a function from B” to B where 
B= {0,1} 


Boolean algebra: a set B with two binary operations V and A, 
elements 0 and 1, and a complementation operator ~ that 
satisfies the identity, complement, associative, commuta- 
tive, and distributive laws 


literal of the Boolean variable x: either x or x 
minterm of x4, Xz ...,X,,: a Boolean product y; y +: Yp, where 
each y; is either x; or x; 


sum-of-products expansion (or disjunctive normal form): 
the representation of a Boolean function as a disjunction of 
minterms 


functionally complete: a set of Boolean operators is called 
functionally complete if every Boolean function can be rep- 
resented using these operators 
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32. yZ ye YE Vz 


33. Show that products of k literals correspond to 2”-*- 
dimensional subcubes of the n-cube Q,,, where the ver- 
tices of the cube correspond to the minterms represented 
by the bit strings labeling the vertices, as described in 
Example 8 of Section 10.2. 


x | y (orx NAND y): the expression that has the value 0 when 
both x and y have the value 1 and the value 1 otherwise 

x | y (or x NOR y): the expression that has the value 0 when 
either x or y or both have the value 1 and the value 0 other- 
wise 

inverter: a device that accepts the value of a Boolean variable 
as input and produces the complement of the input 


OR gate: a device that accepts the values of two or more 
Boolean variables as input and produces their Boolean sum 
as output 

AND gate: a device that accepts the values of two or more 
Boolean variables as input and produces their Boolean 
product as output 

half adder: a circuit that adds two bits, producing a sum bit 
and a carry bit 


full adder: a circuit that adds two bits and a carry, producing 
a sum bit and a carry bit 

K-map for n variables: a rectangle divided into 2” cells where 
each cell represents a minterm in the variables 


minimization of a Boolean function: representing a Boolean 
function as the sum of the fewest products of literals such 
that these products contain the fewest literals possible 
among all sums of products that represent this Boolean 
function 


implicant of a Boolean function: a product of literals with the 
property that if this product has the value 1, then the value 
of this Boolean function is 1 


prime implicant of a Boolean function: a product of literals 
that is an implicant of the Boolean function and no prod- 
uct obtained by deleting a literal is also an implicant of this 
function 

essential prime implicant of a Boolean function: a prime 
implicant of the Boolean function that must be included in 
a minimization of this function 


don’t care condition: a combination of input values for a cir- 
cuit that is not possible or never occurs 
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RESULTS 


The identities for Boolean algebra (see Table 5 in 
Section 12.1). 


An identity between Boolean functions represented by 
Boolean expressions remains valid when the duals of both 
sides of the identity are taken. 


Review Questions 


1. Define a Boolean function of degree n. 
2. How many Boolean functions of degree two are there? 


3. Give a recursive definition of the set of Boolean expres- 
sions. 


4. a) What is the dual of a Boolean expression? 
b) What is the duality principle? How can it be used to 
find new identities involving Boolean expressions? 


5. Explain how to construct the sum-of-products expansion 
of a Boolean function. 


6. a) What does it mean for a set of operators to be func- 
tionally complete? 
b) Is the set {+, -} functionally complete? 
c) Are there sets of a single operator that are functionally 
complete? 


7. Explain how to build a circuit for a light controlled by two 
switches using OR gates, AND gates, and inverters. 


8. Construct a half adder using OR gates, AND gates, and 
inverters. 


9. Is there a single type of logic gate that can be used to build 
all circuits that can be built using OR gates, AND gates, 
and inverters? 


Supplementary Exercises 


1. For which values of the Boolean variables x, y, and z does 
a) x+y+zZ=xyz? 
b) xy +z) =x+ yz? 
c) XyZz=x+y+2? 

2. Let x and y belong to {0, 1}. Does it necessarily follow 
that x = y if there exists a value z in {0, 1} such that 

b) x+z=y+2? 

d) xļz=yłļ2z? 


a) xz = yz? 
c) x9z=y®2? 
e) x|z=y|z? 
A Boolean function F is called self-dual if and only if 
F(X, ...,X%_) = FQ), re n 
3. Which of these functions are self-dual? 
a) F(x y) =x b) F(x, y) =xy+xy 
c) Fœ y)=x+y d) F y) = xy + Xy 


4. Give an example of a self-dual Boolean function of three 
variables. 


Every Boolean function can be represented by a sum-of- 
products expansion. 

Each of the sets {+, ~} and {-, ~} is functionally complete. 

Each of the sets { | } and { | } is functionally complete. 

The use of K-maps to minimize Boolean expressions. 

The Quine—McCluskey method for minimizing Boolean 
expressions. 


10. a) Explain how K-maps can be used to simplify sum-of- 
products expansions in three Boolean variables. 


b) Use a K-map to simplify the sum-of-products expan- 
sion xyz + xyz + xyZ+xXyz+XyZz. 
11. a) Explain how K-maps can be used to simplify sum-of- 
products expansions in four Boolean variables. 
b) Use a K-map to simplify the sum-of-products expan- 
sion wxyz + wxyz + wxyz + wxyzZ + wxyz + wx yz + 
WXYZ + WXYZ + WXYZ. 


12. a) What is a don’t care condition? 


b) Explain how don’t care conditions can be used to 
build a circuit using OR gates, AND gates, and in- 
verters that produces an output of 1 if a decimal digit 
is 6 or greater, and an output of 0 if this digit is less 
than 6. 


13. a) Explain how to use the Quine—McCluskey method to 
simplify sum-of-products expansions. 


b) Use this method to simplify xyz + xyz + xyz + xyz + 


XYZ. 


*5. How many Boolean functions of degree n are self-dual? 
We define the relation < on the set of Boolean functions of 
degree n so that F < G, where F and G are Boolean functions 
if and only if G(x, Xz ...,x,) = 1 whenever F(x, Xz, ..., 
x,) = 1. 
6. Determine whether F < G or G < F for the following 
pairs of functions. 
a) F(x, y)=%G@y)=x+y 
b) F(x, y) =x + y, GG, y) = xy 
o Fy) =%G@y)=xt+y 
7. Show that if F and G are Boolean functions of degree n, 
then 


a) F<F+G. b) FG <F. 


8. Show that if F, G, and H are Boolean functions of degree 
n, then F +G < H if and only if F < H and G < H. 


*9, Show that the relation < is a partial ordering on the set of 
Boolean functions of degree n. 


*10. Draw the Hasse diagram for the poset consisting of the 
set of the 16 Boolean functions of degree two (shown in 
Table 3 of Section 12.1) with the partial ordering <. 


*11. For each of these equalities either prove it is an identity 
or find a set of values of the variables for which it does 
not hold. 


a x|OlD=@ly)1z 
b) xlLoloda=Qlyl@ala 


OxlOla=alwylalz 
Define the Boolean operator © as follows:101=1,100= 


0,001=0,and000=1. 
12. Show that x © y = xy + xy. 


13. Show that x © y = (x @® y). 


14. Show that each of these identities holds. 
a) xOx=1 b) xOx=0 


c) xOy=yOx 
15. Is it always true that (xO y) Oz =x0 0 0z)? 


*16. Determine whether the set {©} is functionally complete. 


*17. How many of the 16 Boolean functions in two variables 
x and y can be represented using only the given set of op- 
erators, variables x and y, and values 0 and 1? 

a{} b){} © {+} æ {,+} 
The notation for an XOR gate, which produces the output 
x @ y from x and y, is as follows: 


=D 


18. Determine the output of each of these circuits. 


. Lip. 


LD 


— ID 
ican y 
ee >— 
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19. Show how a half adder can be constructed using fewer 
gates than are used in Figure 8 of Section 12.3 when XOR 
gates can be used in addition to OR gates, AND gates, and 
inverters. 


20. Design a circuit that determines whether three or more 
of four individuals on a committee vote yes on an issue, 
where each individual uses a switch for the voting. 


A threshold gate produces an output y that is either 0 
or | given a set of input values for the Boolean variables 
Xp Xz ...,X,.A threshold gate has a threshold value T, which 
is a real number, and weights w,, wə, ..., W„, each of which 
is a real number. The output y of the threshold gate is 1 if 
and only if w,x,; + WX + +++ +w,x, 2 T. The threshold gate 
with threshold value T and weights w,, wz, ..., w, iS repre- 
sented by the following diagram. Threshold gates are useful 
in modeling in neurophysiology and in artificial intelligence. 


21. A threshold gate represents a Boolean function. Find a 
Boolean expression for the Boolean function represented 
by this threshold gate. 


xy 


x3 


22. A Boolean function that can be represented by a thresh- 
old gate is called a threshold function. Show that each 
of these functions is a threshold function. 

a) F(x) =x b) F@y)=x+y 
c) Fœ y) = xy d) Fœ y)=x]|y 

e) Fay=xly f) Fœ yz) =x+yz 
g) Fiw, xyz) =w+xy +z 

h) F(w, x, y, Z) = wxz + xyz 


* 23. Show that F(x, y) = x ® y is not a threshold function. 


*24. Show that F(w,x,y,z)=wx+yz is not a threshold 
function. 
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Computer Projects 


Write programs with these input and output. 


*6, 


. Given the values of two Boolean variables x and y, find 


the values of x + y, xy, x ® y, x | y, and x J y. 


. Construct a table listing the set of values of all 256 


Boolean functions of degree three. 


. Given the values of a Boolean function in n variables, 


where is a positive integer, construct the sum-of- 
products expansion of this function. 


. Given the table of values of a Boolean function, express 


this function using only the operators - and T 


. Given the table of values of a Boolean function, express 


this function using only the operators + and , 


Given the table of values of a Boolean function, express 
this function using only the operator | . 


Computations and Explorations 


#7. 


** 10. 


11. 


12. 


Given the table of values of a Boolean function, express 
this function using only the operator |. 


. Given the table of values of a Boolean function of degree 


three, construct its K-map. 


. Given the table of values of a Boolean function of degree 


four, construct its K-map. 

Given the table of values of a Boolean function, use the 
Quine—McCluskey method to find a minimal sum-of- 
products representation of this function. 

Given a threshold value and a set of weights for a thresh- 
old gate and the values of the n Boolean variables in the 
input, determine the output of this gate. 

Given a positive integer n, construct a random Boolean 
expression in n variables in disjunctive normal form. 


Use a computational program or programs you have written to do these exercises. 


w 


. Compute the number of Boolean functions of degrees 


seven, eight, nine, and ten. 


. Construct a table of the Boolean functions of degree three. 
. Construct a table of the Boolean functions of degree four. 
. Express each of the different Boolean expressions in three 


variables in disjunctive normal form with just the NAND 
operator, using as few NAND operators as possible. What 
is the largest number of NAND operators required? 


. Express each of the different Boolean expressions in dis- 


junctive normal form in four variables using just the NOR 


Writing Projects 


Respond to these with essays using outside sources. 


1. Describe some of the early machines devised to solve 


problems in logic, such as the Stanhope Demonstrator, 
Jevons’s Logic Machine, and the Marquand Machine. 


. Explain the difference between combinational circuits 


and sequential circuits. Then explain how flip-flops are 
used to build sequential circuits. 


. Define a shift register and discuss how shift registers are 


used. Show how to build shift registers using flip-flops 
and logic gates. 


4. Show how multipliers can be built using logic gates. 


operator, with as few NOR operators as possible. What is 
the largest number of NOR operators required? 


. Randomly generate 10 different Boolean expressions in 


four variables and determine the average number of steps 
required to minimize them using the Quine—McCluskey 
method. 


. Randomly generate 10 different Boolean expressions in 


five variables and determine the average number of steps 
required to minimize them using the Quine—McCluskey 
method. 


. Find out how logic gates are physically constructed. Dis- 


cuss whether NAND and NOR gates are used in building 
circuits. 


. Explain how dependency notation can be used to describe 


complicated switching circuits. 


. Describe how multiplexers are used to build switching 


circuits. 


. Explain the advantages of using threshold gates to con- 


struct switching circuits. Illustrate this by using threshold 
gates to construct half and full adders. 


9. 


10. 


11. 


Describe the concept of hazard-free switching circuits 
and give some of the principles used in designing such 
circuits. 


Explain how to use K-maps to minimize functions of six 
variables. 

Discuss the ideas used by newer methods for minimiz- 
ing Boolean functions, such as Espresso. Explain how 
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these methods can help solve minimization problems in 
as many as 25 variables. 


Describe what is meant by the functional decomposition 
of a Boolean function of n variables and discuss proce- 
dures for decomposing Boolean functions into a compo- 
sition of Boolean functions with fewer variables. 
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and 
Grammars 
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Machines 
with Output 


13.3 Finite-State 
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13.4 Language 
Recognition 


13.5 Turing 
Machines 


Modeling Computation 


C omputers can perform many tasks. Given a task, two questions arise. The first is: Can 
it be carried out using a computer? Once we know that this first question has an affir- 
mative answer, we can ask the second question: How can the task be carried out? Models of 
computation are used to help answer these questions. 

We will study three types of structures used in models of computation, namely, grammars, 
finite-state machines, and Turing machines. Grammars are used to generate the words of a lan- 
guage and to determine whether a word is in a language. Formal languages, which are generated 
by grammars, provide models both for natural languages, such as English, and for programming 
languages, such as Pascal, Fortran, Prolog, C, and Java. In particular, grammars are extremely 
important in the construction and theory of compilers. The grammars that we will discuss were 
first used by the American linguist Noam Chomsky in the 1950s. 

Various types of finite-state machines are used in modeling. All finite-state machines have 
a set of states, including a starting state, an input alphabet, and a transition function that assigns 
a next state to every pair of a state and an input. The states of a finite-state machine give it 
limited memory capabilities. Some finite-state machines produce an output symbol for each 
transition; these machines can be used to model many kinds of machines, including vending 
machines, delay machines, binary adders, and language recognizers. We will also study finite- 
state machines that have no output but do have final states. Such machines are extensively used 
in language recognition. The strings that are recognized are those that take the starting state 
to a final state. The concepts of grammars and finite-state machines can be tied together. We 
will characterize those sets that are recognized by a finite-state machine and show that these are 
precisely the sets that are generated by a certain type of grammar. 

Finally, we will introduce the concept of a Turing machine. We will show how Turing 
machines can be used to recognize sets. We will also show how Turing machines can be used 
to compute number-theoretic functions. We will discuss the Church-Turing thesis, which states 
that every effective computation can be carried out using a Turing machine. We will explain 
how Turing machines can be used to study the difficulty of solving certain classes of problems. 
In particular, we will describe how Turing machines are used to classify problems as tractable 
versus intractable and solvable versus unsolvable. 


Languages and Grammars 


13.1.1 Introduction 


Words in the English language can be combined in various ways. The grammar of English tells 
us whether a combination of words is a valid sentence. For instance, the frog writes neatly is 
a valid sentence, because it is formed from a noun phrase, the frog, made up of the article the 
and the noun frog, followed by a verb phrase, writes neatly, made up of the verb writes and the 
adverb neatly. We do not care that this is a nonsensical statement, because we are concerned 
only with the syntax, or form, of the sentence, and not its semantics, or meaning. We also note 
that the combination of words swims quickly mathematics is not a valid sentence because it does 
not follow the rules of English grammar. 

The syntax of a natural language, that is, a spoken language, such as English, French, 
German, or Spanish, is extremely complicated. In fact, it does not seem possible to specify all 
the rules of syntax for a natural language. Research in the automatic translation of one language 
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to another has led to the concept of a formal language, which, unlike a natural language, is 
specified by a well-defined set of rules of syntax. Rules of syntax are important not only in 
linguistics, the study of natural languages, but also in the study of programming languages. 

We will describe the sentences of a formal language using a grammar. The use of grammars 
helps when we consider the two classes of problems that arise most frequently in applications 
to programming languages: (1) How can we determine whether a combination of words is a 
valid sentence in a formal language? (2) How can we generate the valid sentences of a formal 
language? Before giving a technical definition of a grammar, we will describe an example of 
a grammar that generates a subset of English. This subset of English is defined using a list of 
rules that describe how a valid sentence can be produced. We specify that 


1. asentence is made up of a noun phrase followed by a verb phrase; 


2. anoun phrase is made up of an article followed by an adjective followed by a noun, 
or 


. anoun phrase is made up of an article followed by a noun; 
. a verb phrase is made up of a verb followed by an adverb, or 
. averb phrase is made up of a verb; 


3 

4 

5 

6. an article is a, or 
7. an article is the; 

8. an adjective is large, or 
9. an adjective is hungry; 
10. anoun is rabbit, or 

11. anoun is mathematician; 
12. a verb is eats, or 

13. a verb is hops; 

14. an adverb is quickly, or 
15. an adverb is wildly. 


From these rules we can form valid sentences using a series of replacements until no more rules 
can be used. For instance, we can follow the sequence of replacements: 


sentence 

noun phrase verb phrase 

article adjective noun verb phrase 
article adjective noun verb adverb 
the adjective noun verb adverb 

the large noun verb adverb 

the large rabbit verb adverb 

the large rabbit hops adverb 

the large rabbit hops quickly 


to obtain a valid sentence. It is also easy to see that some other valid sentences are: a hungry 
mathematician eats wildly, a large mathematician hops, the rabbit eats quickly, and so on. Also, 
we can see that the quickly eats mathematician is not a valid sentence. 


13.1.2 Phrase-Structure Grammars 


Before we give a formal definition of a grammar, we introduce a little terminology. 


Definition 1 


The notion of a 
phrase-structure 
grammar extends 
the concept of a 
rewrite system 
devised by Axel 
Thue in the early 
20th century. 


Definition 2 


EXAMPLE 1 


Definition 3 
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A vocabulary (or alphabet) V is a finite, nonempty set of elements called symbols. A word 
(or sentence) over V is a string of finite length of elements of V. The empty string or null 
string, denoted by A (and sometimes by e), is the string containing no symbols. The set of all 
words over V is denoted by V*. A language over V is a subset of V*. 


Note that A, the empty string, is the string containing no symbols. It is different from Ø, the 
empty set. It follows that {4} is the set containing exactly one string, namely, the empty string. 

Languages can be specified in various ways. One way is to list all the words in the lan- 
guage. Another is to give some criteria that a word must satisfy to be in the language. In this 
section, we describe another important way to specify a language, namely, through the use of a 
grammar, such as the set of rules we gave in the introduction to this section. A grammar pro- 
vides a set of symbols of various types and a set of rules for producing words. More precisely, 
a grammar has a vocabulary V, which is a set of symbols used to derive members of the lan- 
guage. Some of the elements of the vocabulary cannot be replaced by other symbols. These are 
called terminals, and the other members of the vocabulary, which can be replaced by other sym- 
bols, are called nonterminals. The sets of terminals and nonterminals are usually denoted by T 
and N, respectively. In the example given in the introduction of the section, the set of termi- 
nals is {a, the, rabbit, mathematician, hops, eats, quickly, wildly}, and the set of nonterminals 
is {sentence, noun phrase, verb phrase, adjective, article, noun, verb, adverb}. There is a 
special member of the vocabulary called the start symbol, denoted by S, which is the element 
of the vocabulary that we always begin with. In the example in the introduction, the start symbol 
is sentence. The rules that specify when we can replace a string from V*, the set of all strings of 
elements in the vocabulary, with another string are called the productions of the grammar. We 
denote by zo — z, the production that specifies that z) can be replaced by z, within a string. The 
productions in the grammar given in the introduction of this section were listed. The first pro- 
duction, written using this notation, is sentence > noun phrase verb phrase. We summarize 
this terminology in Definition 2. 


A phrase-structure grammar G=(V,T,S,P) consists of a vocabulary V, a subset T 
of V consisting of terminal symbols, a start symbol S from V, and a finite set of produc- 
tions P. The set V — T is denoted by N. Elements of N are called nonterminal symbols. Every 
production in P must contain at least one nonterminal on its left side. 


Let G = (V, T, S, P), where V = {a, b, A, B, S}, T = {a, b}, S is the start symbol, and P = {S > 
ABa, A > BB, B — ab, AB > b}. Gis an example of a phrase-structure grammar. < 


We will be interested in the words that can be generated by the productions of a phrase- 
structure grammar. 


Let G = (V, T, S, P) be a phrase-structure grammar. Let wọ = lzor (that is, the concatena- 
tion of l, z), and r) and w; =/z,r be strings over V. If z) > zı is a production of G, 
we say that w, is directly derivable from wọ and we write wọ > Ww. If Wo, Wi, ..., W, are 
strings over V such that wọ =w1, W1 > Wo, ..-, W,-; > Wp then we say that w, is derivable 
from wo, and we write Wy = w,,. The sequence of steps used to obtain w, from w is called a 
derivation. 
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EXAMPLE 2 


Definition 4 


EXAMPLE 3 


Extra > 
Examples 


EXAMPLE 4 


EXAMPLE 5 


The string Aaba is directly derivable from ABa in the grammar in Example 1 because 
B > ab is a production in the grammar. The string abababa is derivable from ABa because 
ABa = Aaba = BBaba = Bababa = abababa, using the productions B > ab, A > BB, B > ab, 
and B —> ab in succession. < 


Let G = (V, T, S, P) be a phrase-structure grammar. The language generated by G (or the 
language of G), denoted by L(G), is the set of all strings of terminals that are derivable from 
the starting state S. In other words, 


L(G) ={weT*|S3w}. 


In Examples 3 and 4 we find the language generated by a phrase-structure grammar. 


Let G be the grammar with vocabulary V = {S, A, a, b}, set of terminals T = {a, b}, starting 
symbol S, and productions P = {S > aA, S > b, A > aa}. What is L(G), the language of this 
grammar? 


Solution: From the start state S we can derive aA using the production S > aA. We can 
also use the production S$ > b to derive b. From aA the production A > aa can be used to 
derive aaa. No additional words can be derived. Hence, L(G) = {b, aaa}. < 


Let G be the grammar with vocabulary V = {S, 0, 1}, set of terminals T = {0, 1}, starting sym- 
bol S, and productions P = {S > 11S, S > 0}. What is L(G), the language of this grammar? 


Solution: From S we can derive 0 using S > 0, or 11S using S > 11S. From 11S we can derive 
either 110 or 1111S. From 1111S we can derive 11110 and 1111115. At any stage of a deriva- 
tion we can either add two 1s at the end of the string or terminate the derivation by adding a 
0 at the end of the string. We surmise that L(G) = {0, 110, 11110, 1111110, ... }, the set of all 
strings that begin with an even number of 1s and end with a 0. This can be proved using an 
inductive argument that shows that after n productions have been used, the only strings of ter- 
minals generated are those consisting of n — 1 concatenations of 11 followed by 0. (This is left 
as an exercise for the reader.) < 


The problem of constructing a grammar that generates a given language often arises. Ex- 
amples 5, 6, and 7 describe problems of this kind. 


Give a phrase-structure grammar that generates the set {0"1" | n = 0, 1, 2, ... }. 


Solution: Two productions can be used to generate all strings consisting of a string of Os followed 
by a string of the same number of 1s, including the null string. The first builds up successively 
longer strings in the language by adding a 0 at the start of the string and a 1 at the end. The second 
production replaces S with the empty string. The solution is the grammar G = (V, T, S, P), where 
V = {0, 1, S}, T = {0, 1}, S is the starting symbol, and the productions are 


S —> 0S1 
S> À. 


The verification that this grammar generates the correct set is left as an exercise for the 
reader. < 


EXAMPLE 6 


EXAMPLE 7 


Links » 
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Example 5 involved the set of strings made up of Os followed by 1s, where the number 
of Os and 1s are the same. Example 6 considers the set of strings consisting of Os followed 
by 1s, where the number of Os and 1s may differ. 


Find a phrase-structure grammar to generate the set {0’"1" | m and n are nonnegative integers}. 


Solution: We will give two grammars G, and G, that generate this set. This will illustrate that 
two different grammars can generate the same language. 

The grammar G; has alphabet V = {S, 0, 1}; terminals T = {0, 1}; and productions S$ > OS, 
S — S1, and S > A. Then, G, generates the correct set, because using the first production m 
times puts m Os at the beginning of the string, and using the second production n times puts n 
1s at the end of the string. Furthermore, every string produced by this grammar is of the form 
01”, because the only way to add a symbol to the beginning of a string is to add a 0 by applying 
the production S — OS, and the only way to add a symbol to the end of the string is to add a 1 
by applying the production $ > S1. 

The grammar G, has alphabet V = {S,A,0, 1}; terminals T = {0,1}; and productions 
S > 0S, S > 1A, S > 1, A > 1A,A > 1, and S > A. The details that this grammar generates 
the correct set are left as an exercise for the reader. < 


Sometimes a set that is easy to describe can be generated only by a complicated grammar. 
Example 7 illustrates this. 


One grammar that generates the set {0"1"2"|n=0,1,2,3,...} is G=(V,T7S,P) 
with V = {0,1,2,5,A,B,C}; T= {0,1,2}, starting state S; and productions S$ > C, 
C > OCAB, S > A, BA —> AB, OA > 01, 1A — 11, 1B > 12, and 2B > 22. We leave it 
as an exercise for the reader (Exercise 12) to show that this statement is correct. The grammar 
given is the simplest type of grammar that generates this set, in a sense that will be made clear 
later in this section. < 


13.1.3 Types of Phrase-Structure Grammars 


Phrase-structure grammars can be classified according to the types of productions that are 
allowed. We will describe the classification scheme introduced by Noam Chomsky. In Sec- 
tion 13.4 we will see that the different types of languages defined in this scheme correspond to 
the classes of languages that can be recognized using different models of computing machines. 

A type 0 grammar has no restrictions on its productions. A type 1 grammar can have pro- 
ductions of the form w; > wz, where w; = lAr and w, = lwr, where A is a nonterminal sym- 
bol, / and r are strings of zero or more terminal or nonterminal symbols, and w is a nonempty 
string of terminal or nonterminal symbols. It can also have the production S > A as long 
as S does not appear on the right-hand side of any other production. A type 2 grammar can 
have productions only of the form w, —> w,, where w, is a single symbol that is not a terminal 
symbol. A type 3 grammar can have productions only of the form w, > w, with w, = A and 
either w, = aB or w, = a, where A and B are nonterminal symbols and a is a terminal symbol, 
or with w, = S and w, = A. 

Type 2 grammars are called context-free grammars because a nonterminal symbol that is 
the left side of a production can be replaced in a string whenever it occurs, no matter what else 
is in the string. A language generated by a type 2 grammar is called a context-free language. 
When there is a production of the form /w,r > lw r (but not of the form w} > w3), the grammar 
is called type 1 or context-sensitive because w, can be replaced by w, only when it is surrounded 
by the strings / and r. A language generated by a type 1 grammar is called a context-sensitive 
language. Type 3 grammars are also called regular grammars. A language generated by a 
regular grammar is called regular. Section 13.4 deals with the relationship between regular 
languages and finite-state machines. 
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EXAMPLE 8 
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EXAMPLE 9 


EXAMPLE 10 


Of the four types of grammars we have defined, context-sensitive grammars have the most 
complicated definition. Sometimes, these grammars are defined in a different way. A production 
of the form w, > w, is called noncontracting if the length of w, is less than or equal to the 
length of w,. According to our characterization of context-senstive languages, every produc- 
tion in a type 1 grammar, other than the production S$ — A, if it is present, is noncontracting. It 
follows that the lengths of the strings in a derivation in a context-sensitive language are non- 
decreasing unless the production S > A is used. This means that the only way for the empty 
string to belong to the language generated by a context-sensitive grammar is for the production 
S — Ato be part of the grammar. The other way that context-sensitive grammars are defined is 
by specifying that all productions are noncontracting. A grammar with this property is called 
noncontracting or monotonic. The class of noncontracting grammars is not the same as the 
class of context-sensitive grammars. However, these two classes are closely related; it can be 
shown that they define the same set of languages except that noncontracting grammars cannot 
generate any language containing the empty string A. 


From Example 6 we know that {0’"1” | m, n = 0, 1, 2, ... } is a regular language, because it can 
be generated by a regular grammar, namely, the grammar G, in Example 6. < 


Context-free and regular grammars play an important role in programming languages. 
Context-free grammars are used to define the syntax of almost all programming languages. 
These grammars are strong enough to define a wide range of languages. Furthermore, efficient 
algorithms can be devised to determine whether and how a string can be generated. Regular 
grammars are used to search text for certain patterns and in lexical analysis, which is the pro- 
cess of transforming an input stream into a stream of tokens for use by a parser. 


It follows from Example 5 that {0”1” | n = 0, 1, 2, ... } is a context-free language, because the 
productions in this grammar are S$ > 0S1 and S > A. However, it is not a regular language. This 
will be shown in Section 13.4. < 


The set {0"1"2" | n = 0, 1, 2, ... } is a context-sensitive language, because it can be generated 
by a type 1 grammar, as Example 7 shows, but not by any type 2 language. (This is shown in 
Exercise 28 in the supplementary exercises at the end of the chapter.) < 


Table 1 summarizes the terminology used to classify phrase-structure grammars. 


13.1.4 Derivation Trees 


A derivation in the language generated by a context-free grammar can be represented graphically 
using an ordered rooted tree, called a derivation, or parse tree. The root of this tree represents 
the starting symbol. The internal vertices of the tree represent the nonterminal symbols that 


TABLE 1 Types of Grammars. 


Type Restrictions on Productions w, > w, 


No restrictions 
w; = lAr and w, = lwr, where A E€ N, l, r,w E€ (N U T)* and w F A; 


or w, = S and w, = Aas long as S is not on the right-hand side of another production 


w, =A, where A is a nonterminal symbol 
w; = A and w, = aB or w, =a, where A E€ N, B E€ N, anda E T; orw, = Sand w, = A 


EXAMPLE 11 


EXAMPLE 12 
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sentence 
ae Ae 
noun phrase verb phrase 
article adjective noun verb adverb 
the hungry rabbit eats quickly 


FIGURE 1 A derivation tree. 


arise in the derivation. The leaves of the tree represent the terminal symbols that arise. If the 
production A —> w arises in the derivation, where w is a word, the vertex that represents A has 
as children vertices that represent each symbol in w, in order from left to right. 


Construct a derivation tree for the derivation of the hungry rabbit eats quickly, given in the 
introduction of this section. 


Solution: The derivation tree is shown in Figure 1. < 


The problem of determining whether a string is in the language generated by a context-free 
grammar arises in many applications, such as in the construction of compilers. When given a 
string, the naive approach for determining whether it is in the language generated by a grammar 
is to look for a sequence of productions that can be applied, beginning at the start state, that 
lead to the given string. When following such an approach, it is useful to think a few moves 
ahead. This approach is known as top-down parsing. A second approach, known as bottom-up 
parsing, is to work backward from the given string with the goal of undoing productions one-by- 
one to reach the start symbol. We illustrate these two approaches to this problem in Example 12. 


Determine whether the word cbhab belongs to the language generated by the grammar G = 
(V, T, S, P), where V = {a, b, c, A, B, C, S}, T = {a, b, c}, S is the starting symbol, and the pro- 
ductions are 


S > AB 
A > Ca 
B > Ba 
B > Cb 
Bob 
C-> cb 
C >b. 


Solution: A top-down approach to this problem is to begin with S and attempt to derive cbab 
using a series of productions. Because there is only one production with S on its left-hand side, 
we must start with S=+ AB. Next we use the only production that has A on its left-hand side, 


AVRAM NOAM CHOMSKY (BORN 1928) Noam Chomsky, born in Philadelphia, is the son of a Hebrew 
scholar. He received his B.A., M.A., and Ph.D. in linguistics, all from the University of Pennsylvania. He was 
on the staff of the University of Pennsylvania from 1950 until 1951. In 1955 he joined the faculty at M.LT., 
beginning his M.LT. career teaching engineers French and German. Chomsky is currently the Ferrari P. Ward 
Professor of foreign languages and linguistics at M.I.T. He is known for his many fundamental contributions 
to linguistics, including the study of grammars. Chomsky is also widely known for his outspoken political 
activism. 
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namely, A > Ca, to obtain S= AB = CaB. Because cbab begins with the symbols cb, we use 
the production C > cb. This gives us S + AB = CaB = cbaB. We finish by using the production 
B > b, to obtain S= AB = CaB = cbaB = cbab. 

We will solve the same problem using bottom-up parsing. Because cbab is the string to be 
derived, we can use the production C > cb, so that Cab = cbab. Then, we can use the production 
A > Ca, so that Ab = Cab = cbab. Using the production B > b gives AB > Ab = Cab = cbab. 
Finally, using S —> AB shows that a complete derivation for chab is SAB Abs Cab 
cbab. < 


Example 12 presents the solution of a parsing problem using both top-down and bottom-up 
parsing. Both approaches were easy to use for that problem. However, parsing problems can 
be quite challenging. That is, it can be quite challenging to determine whether a string is in the 
language generated by a context-free grammar. Because parsing is so important, many strategies 
and algorithms have been devised for top-down and for bottom-up parsing. These algorithms 
are beyond the scope of this book. The interested reader should consult [AhLaSeUl106] to learn 
more. 


13.1.5 Backus-Naur Form 


There is another notation that is sometimes used to specify a type 2 grammar, called the 
Backus-Naur form (BNF), after John Backus, who invented it, and Peter Naur, who refined 


JOHN BACKUS (1924-2007) John Backus was born in Philadelphia and grew up in Wilmington, Delaware. 
He attended the Hill School in Pottstown, Pennsylvania. He needed to attend summer school every year because 
he disliked studying and was not a serious student. But he enjoyed spending his summers in New Hampshire 
where he attended summer school and amused himself with summer activities, including sailing. He obliged 
his father by enrolling at the University of Virginia to study chemistry. But he quickly decided chemistry was 
not for him, and in 1943 he entered the army, where he received medical training and worked in a neuro- 
surgery ward in an army hospital. Ironically, Backus was soon diagnosed with a bone tumor in his skull and 
was fitted with a metal plate. His medical work in the army convinced him to try medical school, but he aban- 
doned this after nine months because he disliked the rote memorization required. After dropping out of medical 
school, he entered a school for radio technicians because he wanted to build his own high fidelity set. A teacher 
in this school recognized his potential and asked him to help with some mathematical calculations needed for an article in a magazine. 
Finally, Backus found what he was interested in: mathematics and its applications. He enrolled at Columbia University, from which 
he received both bachelor’s and master’s degrees in mathematics. Backus joined IBM as a programmer in 1950. He participated in 
the design and development of two of IBM’s early computers. From 1954 to 1958 he led the IBM group that developed FORTRAN. 
Backus became a staff member at the IBM Watson Research Center in 1958. He was part of the committees that designed the 
programming language ALGOL using what is now called the Backus—Naur form for the description of the syntax of this language. 
(The development of ALGOL led to the development of other programming languages, including Pascal and C.) Later, Backus 
worked on the mathematics of families of sets and on a functional style of programming. Backus became an IBM Fellow in 1963, 
and he received the National Medal of Science in 1974 and the prestigious Turing Award from the Association for Computing 
Machinery in 1977. 


Courtesy of Louis Bachrach 


PETER NAUR (1928-2016) Peter Naur was born in Frederiksberg, near Copenhagen. As a boy he became 
interested in astronomy. Not only did he observe heavenly bodies, but he also computed the orbits of comets 
and asteroids. Naur attended Copenhagen University, receiving his degree in 1949. He spent 1950 and 1951 in 
Cambridge, where he used an early computer to calculate the motions of comets and planets. After returning to 
Denmark, he continued working in astronomy but kept his ties to computing. In 1955 he served as a consultant 
to the building of the first Danish computer. In 1959 Naur made the switch from astronomy to computing as 
a full-time activity. His first job as a full-time computer scientist was participating in the development of the 
programming language ALGOL. From 1960 to 1967 he worked on the development of compilers for ALGOL 
and COBOL. From 1969 until 1999 he was professor of computer science at Copenhagen University, where 


@©Heidelberg Laureate . . j Š Zs H 
Forum Foundation he has worked in the area of programming methodology. His research interests included the design, structure, 


and performance of computer programs. Naur was a pioneer in both the areas of software architecture and software engineering. He 
rejected the view that computer programming is a branch of mathematics and preferred that computer science be called datalogy. 


The ancient Indian 
grammarian Panini 
specified Sanskrit using 
3959 rules; 
Backus-Naur form is 
sometimes called 
Backus-Panini form. 
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it for use in the specification of the programming language ALGOL. (Surprisingly, a notation 
quite similar to the Backus—Naur form was used approximately 2500 years ago to describe the 
grammar of Sanskrit.) The Backus—Naur form is used to specify the syntactic rules of many 
computer languages, including Java. The productions in a type 2 grammar have a single non- 
terminal symbol as their left-hand side. Instead of listing all the productions separately, we can 
combine all those with the same nonterminal symbol on the left-hand side into one statement. 
Instead of using the symbol — in a production, we use the symbol ::=. We enclose all nonter- 
minal symbols in brackets, ( }, and we list all the right-hand sides of productions in the same 
statement, separating them by bars. For instance, the productions A > Aa, A > a, and A > AB 
can be combined into (A) ::= (A)a | a | (A)(B). 

Example 13 illustrates how the Backus—Naur form is used to describe the syntax of pro- 
gramming languages. Our example comes from the original use of Backus—Naur form in the 
description of ALGOL 60. 


In ALGOL 60 an identifier (which is the name of an entity such as a variable) consists of a string 
of alphanumeric characters (that is, letters and digits) and must begin with a letter. We can use 
these rules in Backus—Naur to describe the set of allowable identifiers: 


(identifier) ::= (letter) | (identifier) (letter) | (identifier) (digit) 
(letter) ::=a|b|--+|y|2z__ the ellipsis indicates that all 26 letters are included 
(digit) :=O|1|/2|/3]4|/5|]6|7|8|9 


For example, we can produce the valid identifier x99a by using the first rule to replace 
(identifier) by (identifier) (letter), the second rule to obtain (identifier)a, the first rule twice 
to obtain (identifier) (digit) (digit)a, the third rule twice to obtain (identifier)99a, the first rule 
to obtain (/etter)99a, and finally the second rule to obtain x99a. < 


What is the Backus—Naur form of the grammar for the subset of English described in the intro- 
duction to this section? 


Solution: The Backus—Naur form of this grammar is 


(sentence) ::= (noun phrase) (verb phrase) 

(noun phrase) ::= (article) (adjective)(noun) | (article){noun) 
(verb phrase) ::= (verb)(adverb) | (verb) 

(article) ::= a | the 

(adjective) ::= large | hungry 


(noun) ::= rabbit | mathematician 
(verb) ::= eats | hops 
(adverb) ::= quickly | wildly < 


Give the Backus—Naur form for the production of signed integers in decimal notation. (A signed 
integer is a nonnegative integer preceded by a plus sign or a minus sign.) 


Solution: The Backus—Naur form for a grammar that produces signed integers is 


(signed integer) ::= (sign) (integer 

(sign) := + | — 

(integer) ::= (digit) | (digit) (integer) 

(digit) ::=0|1|/2]3141516/718]9 < 
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The Backus-Naur form, with a variety of extensions, is used extensively to specify the 
syntax of programming languages, such as Java and LISP; database languages, such as SQL; 
and markup languages, such as XML. Some extensions of the Backus—Naur form that are com- 
monly used in the description of programming languages are introduced in the preamble to 


Exercise 34. 


Exercises 


Exercises 1-3 refer to the grammar with start symbol sen- 
tence, set of terminals T = {the, sleepy, happy, tortoise, hare, 
Passes, runs, quickly, slowly}, set of nonterminals N = {noun 
phrase, transitive verb phrase, intransitive verb phrase, 
article, adjective, noun, verb, adverb}, and productions: 


sentence — noun phrase transitive verb phrase 
noun phrase 

sentence —> noun phrase intransitive verb phrase 

noun phrase — article adjective noun 

noun phrase — article noun 

transitive verb phrase — transitive verb 

intransitive verb phrase — intransitive verb adverb 

intransitive verb phrase — intransitive verb 

article > the 


adjective —> sleepy 
adjective > happy 

noun > tortoise 

noun > hare 

transitive verb > passes 
intransitive verb — runs 
adverb —> quickly 
adverb —> slowly 


1. Use the set of productions to show that each of these sen- 
tences is a valid sentence. 


a) the happy hare runs 
b) the sleepy tortoise runs quickly 
c) the tortoise passes the hare 
d) the sleepy hare passes the happy tortoise 
2. Find five other valid sentences, besides those given in 
Exercise 1. 


3. Show that the hare runs the sleepy tortoise is not a valid 
sentence. 


4. Let G=(V,TS, P) be the phrase-structure grammar 
with V = {0,1,A,S}, T = {0,1}, and set of produc- 
tions P consisting of S> 1S, S — 00A, A > OA, 
andA > 0. 

a) Show that 111000 belongs to the language generated 
by G. 

b) Show that 11001 does not belong to the language gen- 
erated by G. 


c) What is the language generated by G? 


5. 


*6, 


10. 


11. 


*12. 


13. 


14. 


Let G = (V, T, S, P) be the phrase-structure grammar with 

V = {0, 1,A, B, S}, T = {0,1}, and set of productions 

P consisting of S > 0A, S —> 1A, A > OB, B > 1A, 

Bol. 

a) Show that 10101 belongs to the language generated 
by G. 

b) Show that 10110 does not belong to the language gen- 
erated by G. 

c) What is the language generated by G? 

Let V = {S, A, B, a, b} and T = {a, b}. Find the language 

generated by the grammar (V, T, S, P) when the set P of 

productions consists of 

a) S > AB, A > ab, B > bb. 

b) S ~ AB, S > aA, A > a, B > ba. 

c) S —> AB, S > AA, A > aB, A > ab, B > b. 

d) S =~ AA, S > B, A > aaA, A > aa, B > bB, B = b. 

e) S — AB, A > aAb, B > bBa, A > À, B > å. 


. Construct a derivation of 0°1° using the grammar given 


in Example 5. 


. Show that the grammar given in Example 5 generates the 


set {0"1” |n =0,1,2,... }. 


. a) Construct a derivation of 0714 using the grammar G; 


in Example 6. 

b) Construct a derivation of 071+ using the grammar G, 
in Example 6. 

a) Show that the grammar G, given in Example 6 gen- 
erates the set {0’"1" | m,n = 0, 1, 2,... }. 

b) Show that the grammar G, in Example 6 generates 
the same set. 

Construct a derivation of 07172? in the grammar given in 

Example 7. 

Show that the grammar given in Example 7 generates the 

set {071"2" |n=0, 1, 2,... }. 

Find a phrase-structure grammar for each of these lan- 

guages. 

a) the set consisting of the bit strings 0, 1, and 11 

b) the set of bit strings containing only 1s 

c) the set of bit strings that start with 0 and end with 1 

d) the set of bit strings that consist of a 0 followed by an 
even number of 1s 

Find a phrase-structure grammar for each of these lan- 

guages. 

a) the set consisting of the bit strings 10, 01, and 101 

b) the set of bit strings that start with 00 and end with 
one or more Is 

c) the set of bit strings consisting of an even number 
of 1s followed by a final 0 


#15, 


16. 


17. 


18. 


19. 


20. 


d) the set of bit strings that have neither two consecutive 
Os nor two consecutive 1s 


Find a phrase-structure grammar for each of these lan- 

guages. 

a) the set of all bit strings containing an even number 
of Os and no 1s 

b) the set of all bit strings made up of a | followed by an 
odd number of Os 

c) the set of all bit strings containing an even number 
of Os and an even number of 1s 

d) the set of all strings containing 10 or more Os and no 
Is 

e) the set of all strings containing more Os than 1s 

f) the set of all strings containing an equal number of Os 
and Is 


g) the set of all strings containing an unequal number 
of Os and 1s 


Construct phrase-structure grammars to generate each of 
these sets. 

a) {1"|n>0} 

ce) {(11)"|n>0} 


b) {10" |n > 0} 


Construct phrase-structure grammars to generate each of 
these sets. 

a) {0 |n 20} 

c) {(000)" |n > 0} 


b) {1"0 |n > 0} 


Construct phrase-structure grammars to generate each of 
these sets. 

a) {01” |n > 0} 

b) {01” |n > 0} 

c) {0"1”0" |m > 0 andn > 0} 


Let V={S,A,B,a,b} and T= {a,b}. Determine 
whether G = (V, T, S, P) is a type 0 grammar but not a 
type 1 grammar, a type 1 grammar but not a type 2 gram- 
mar, or a type 2 grammar but not a type 3 grammar if P, 
the set of productions, is 

a) S > aAB, A > Bb, B > À. 

b) S > aA, A >a,A >b. 

c) S > ABa, AB > a. 

d) S — AMA, A > aB, B > ab. 

e) S> M,A >B,B >a. 

f) S> A, a > B, B > A,A >b. 

g) S> M,A >b,S > å. 

h) S — AB, B > aAb, Ab > b. 

i) So aA, A > B, B > b,B > å. 

j S>A,A>B,B>åì. 


A palindrome is a string that reads the same backward 
as it does forward, that is, a string w, where w = wË, 
where w” is the reversal of the string w. Find a context- 
free grammar that generates the set of all palindromes 
over the alphabet {0, 1}. 


*21. 


22. 


23. 


24. 


* 25. 


* 26. 
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Let G, and G, be context-free grammars, generating 
the languages L(G,) and L(G), respectively. Show that 
there is a context-free grammar generating each of 
these sets. 


a) L(G,;) UL(G)) 
c) L(G,)* 


b) L(G, )L(G,) 


Find the strings constructed using the derivation trees 
shown here. 


sentence 


a *s 


noun phrase verb phrase 


Zin sS\ 


article adjective noun verb adverb 
a large mathematician hops wildly 
signed integer 
S \ 
sign integer 
/ N 
+ digit integer 
r a 
9 digit integer 
8 digit 
7 


Construct derivation trees for the sentences in Exercise 1. 


Let G be the grammar with V = {a, b,c, S}, T= 
{a,b,c}; starting symbol S, and productions § > 


abS, S > bcS, S > bbS, S > a, and S > cb. Construct 
derivation trees for 
a) bcbba. b) bbbcbba. 


c) bcabbbbbcb. 


Use top-down parsing to determine whether each of the 
following strings belongs to the language generated by 
the grammar in Example 12. 

a) baba b) abab 

c) cbaba d) bbbcba 


Use bottom-up parsing to determine whether the strings 
in Exercise 25 belong to the language generated by the 
grammar in Example 12. 
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27. Construct a derivation tree for —109 using the grammar 


given in Example 15. 


28. a) Explain what the productions are in a grammar if the 


Backus—Naur form for productions is as follows: 


(expression) ::= ((expression)) | 
(expression) + (expression) | 
(expression) * (expression) | 
(variable) 


(variable) ::= x| y 


b) Find a derivation tree for (x * y) + xin this grammar. 


29. a) Construct a phrase-structure grammar that generates 


all signed decimal numbers, consisting of a sign, ei- 
ther + or —; a nonnegative integer; and a decimal frac- 
tion that is either the empty string or a decimal point 
followed by a positive integer, where initial zeros in 
an integer are allowed. 


b) Give the Backus—Naur form of this grammar. 


c) Construct a derivation tree for —31.4 in this grammar. 


30. a) Construct a phrase-structure grammar for the set of all 


fractions of the form a/b, where a is a signed integer 
in decimal notation and b is a positive integer. 


b) What is the Backus—Naur form for this grammar? 


c) Construct a derivation tree for +311/17 in this 
grammar. 


31. Give production rules in Backus-Naur form for an iden- 


tifier if it can consist of 
a) one or more lowercase letters. 


b) at least three but no more than six lowercase letters. 


c) one to six uppercase or lowercase letters beginning 
with an uppercase letter. 


d) a lowercase letter, followed by a digit or an under- 
score, followed by three or four alphanumeric char- 
acters (lower or uppercase letters and digits). 


32. Give production rules in Backus—Naur form for the name 


of a person if this name consists of a first name, which is 
a string of letters, where only the first letter is uppercase; 
a middle initial; and a last name, which can be any string 
of letters. 


33. Give production rules in Backus—Naur form that gener- 


ate all identifiers in the C programming language. In C 
an identifier starts with a letter or an underscore (_) that 
is followed by one or more lowercase letters, uppercase 
letters, underscores, and digits. 


» Several extensions to Backus—Naur form are commonly used 
to define phrase-structure grammars. In one such extension, a 
question mark (?) indicates that the symbol, or group of sym- 
bols inside parentheses, to its left can appear zero or once (that 


is, it is optional), an asterisk (*) indicates that the symbol to 
its left can appear zero or more times, and a plus (+) indi- 
cates that the symbol to its left can appear one or more times. 
These extensions are part of extended Backus-Naur form 
(EBNF), and the symbols ?, *, and + are called metacharac- 
ters. In EBNF the brackets used to denote nonterminals are 
usually not shown. 


34. 


35. 


36. 


37. 


38. 


Describe the set of strings defined by each of these sets 
of productions in EBNF. 
a) string ::= L+D?L+ 
Lasa b le 
Dz=0 |1 
b) string ::= sign D+ | D+ 
sign ::= + | — 
D::=0|1|/2|3|/41516|71819 
c) string ::= L*(D+)?L * 
Lax |y 
D:=0]|1 


Give production rules in extended Backus-Naur form 
that generate all decimal numerals consisting of an op- 
tional sign, a nonnegative integer, and a decimal fraction 
that is either the empty string or a decimal point followed 
by an optional positive integer optionally preceded by 
some number of zeros. 


Give production rules in extended Backus—Naur form 
that generate a sandwich if a sandwich consists of a lower 
slice of bread; mustard or mayonnaise; optional lettuce; 
an optional slice of tomato; one or more slices of either 
turkey, chicken, or roast beef (in any combination); op- 
tionally some number of slices of cheese; and a top slice 
of bread. 


Give production rules in extended Backus—Naur form 
for identifiers in the C programming language (see Ex- 
ercise 33). 


Describe how productions for a grammar in extended 
Backus—Naur form can be translated into a set of produc- 
tions for the grammar in Backus—Naur form. 


This is the Backus—Naur form that describes the syntax of ex- 
pressions in postfix (or reverse Polish) notation. 


39. 


(expression) ::= (term) | (term){term)(addOperator) 
(addOperator) ::= + | — 

(term) ::= (factor) | (factor) (factor) {mulOperator) 
(mulOperator) ::=*| / 

(factor) ::= (identifier) | (expression) 

(identifier) :=a |b |- |z 


For each of these strings, determine whether it is gener- 
ated by the grammar given for postfix notation. If it is, 
find the steps used to generate the string 


a) abc*x+ b) xy++ 
c) xy-z* d) wxyz—*/ 
e) ade-* 
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40. Use Backus—Naur form to describe the syntax of ex- Exercise 40. If it is, find the steps used to generate the 
pressions in infix notation, where the set of operators string. 
and identifiers is the same as in the BNF for postfix a) x+y+z b) a/b+c/d 
expressions given in the preamble to Exercise 39, but c) m*(n+p) d) +m-n+p-q 
parentheses must surround expressions being used as e) (m+n) *(p—q) 
factors. 42. Let G be a grammar and let R be the relation contain- 
ing the ordered pair (wọ, w,) if and only if w; is directly 
41. For each of these strings, determine whether it is derivable from wọ in G. What is the reflexive transitive 
generated by the grammar for infix expressions from closure of R? 


Finite-State Machines with Output 


Links > 


Finite-state machines 
with output are often 
called finite-state 
transducers. 


13.2.1 Introduction 


Many kinds of machines, including components in computers, can be modeled using a structure 
called a finite-state machine. Several types of finite-state machines are commonly used in mod- 
els. All these versions of finite-state machines include a finite set of states, with a designated 
starting state, an input alphabet, and a transition function that assigns a next state to every state 
and input pair. Finite-state machines are used extensively in applications in computer science 
and data networking. For example, finite-state machines are the basis for programs for spell 
checking, grammar checking, indexing or searching large bodies of text, recognizing speech, 
transforming text using markup languages such as XML and HTML, and network protocols 
that specify how computers communicate. 

In this section, we will study those finite-state machines that produce output. We will show 
how finite-state machines can be used to model a vending machine, a machine that delays input, 
a machine that adds integers, and a machine that determines whether a bit string contains a 
specified pattern. 

Before giving formal definitions, we will show how a vending machine can be modeled. A 
vending machine accepts nickels (5 cents), dimes (10 cents), and quarters (25 cents). When a 
total of 30 cents or more has been deposited, the machine immediately returns the amount in 
excess of 30 cents. When 30 cents has been deposited and any excess refunded, the customer 
can push an orange button and receive an orange juice or push a red button and receive an 
apple juice. We can describe how the machine works by specifying its states, how it changes 
states when input is received, and the output that is produced for every combination of input 
and current state. 

The machine can be in any of seven different states s;, i = 0, 1, 2, ...,6, where s; is the state 
where the machine has collected 5i cents. The machine starts in state sọ, with O cents received. 
The possible inputs are 5 cents, 10 cents, 25 cents, the orange button (O), and the red button (R). 
The possible outputs are nothing (n), 5 cents, 10 cents, 15 cents, 20 cents, 25 cents, an orange 
juice, and an apple juice. 

We illustrate how this model of the machine works with this example. Suppose that a student 
puts in a dime followed by a quarter, receives 5 cents back, and then pushes the orange button 
for an orange juice. The machine starts in state sy. The first input is 10 cents, which changes 
the state of the machine to s, and gives no output. The second input is 25 cents. This changes 
the state from s, to sę, and gives 5 cents as output. The next input is the orange button, which 
changes the state from sę back to sọ (because the machine returns to the start state) and gives an 
orange juice as its output. 

We can display all the state changes and output of this machine in a table. To do this we 
need to specify for each combination of state and input the next state and the output obtained. 
Table 1 shows the transitions and outputs for each pair of a state and an input. 

Another way to show the actions of a machine is to use a directed graph with labeled edges, 
where each state is represented by a circle, edges represent the transitions, and edges are labeled 
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Definition 1 


EXAMPLE 1 


TABLE 1 State Table for a Vending Machine. 


Input Input 
State 5 10 25 O R 5 10 25 (0) R 
So 1 S2 S5 So So n n 


Sy 53 S6 Sy 
53 S4 S6 Sy 
S4 S5 S6 S3 
S3 S6 S6 Sy 
S6 S6 S6 55 


S6 S6 S6 So 


with the input and the output for that transition. Figure 1 shows such a directed graph for the 
vending machine. 


13.2.2 Finite-State Machines with Outputs 


We will now give the formal definition of a finite-state machine with output. 


A finite-state machine M = (S, I, O, f, g, Sọ) consists of a finite set S of states, a finite input 
alphabet I, a finite output alphabet O, a transition function f that assigns to each state and 
input pair a new state, an output function g that assigns to each state and input pair an output, 
and an initial state so. 


Let M = (S, L O, f, 8, Sọ) be a finite-state machine. We can use a state table to represent the 
values of the transition function f and the output function g for all pairs of states and input. We 
previously constructed a state table for the vending machine discussed in the introduction to this 
section. 


The state table shown in Table 2 describes a finite-state machine with S = {5p, Sp S% 83}, 
I= {0,1}, and O = {0,1}. The values of the transition function f are displayed in the first 
two columns, and the values of the output function g are displayed in the last two columns. < 


FIGURE 1 A vending machine. 
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FIGURE 2 The state diagram for the 
finite-state machine shown in Table 2. 


Another way to represent a finite-state machine is to use a state diagram, which is a directed 
graph with labeled edges. In this diagram, each state is represented by a circle. Arrows labeled 
with the input and output pair are shown for each transition. 


EXAMPLE 2 Construct the state diagram for the finite-state machine with the state table shown in Table 2. 


Solution: The state diagram for this machine is shown in Figure 2. < 


EXAMPLE 3 Construct the state table for the finite-state machine with the state diagram shown in Figure 3. 


Solution: The state table for this machine is shown in Table 3. < 


An input string takes the starting state through a sequence of states, as determined by the 
transition function. As we read the input string symbol by symbol (from left to right), each input 
symbol takes the machine from one state to another. Because each transition produces an output, 
an input string also produces an output string. 

Suppose that the input string is x = x,x, ...x,. Then, reading this input takes the machine 
from state sọ to state s;, where s; = f (So, x,), then to state s,, where s, = f(s,, X2), and so on, with 
5; = f(s; X;) for j = 1,2,...,k, ending at state s, = f(s,_),x,). This sequence of transitions 
produces an output string yy» ....y,, where y; = 8(Sọ xı) is the output corresponding to the 
transition from sp to S1, Y2 = g(s), X2) is the output corresponding to the transition from s, to sy, 
and so on. In general, y= B(S;-1) x;) forj = 1, 2, ..., k. Hence, we can extend the definition of 


TABLE 3 
Z le 


Input Input 
State 0 1 0 1 
1 53 


Sy So 
S3 S4 


Sy So 


S3 S4 


FIGURE 3 A finite-state machine. 
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EXAMPLE 4 


EXAMPLE 5 


EXAMPLE 6 


? 


the output function g to input strings so that g(x) = y, where y is the output corresponding to 
the input string x. This notation is useful in many applications. 


Find the output string generated by the finite-state machine in Figure 3 if the input string 
is 101011. 


Solution: The output obtained is 001000. The successive states and outputs are shown in 
Table 4. < 


We can now look at some examples of useful finite-state machines. Examples 5, 6, and 7 
illustrate that the states of a finite-state machine give it limited memory capabilities. The states 
can be used to remember the properties of the symbols that have been read by the machine. 
However, because there are only finitely many different states, finite-state machines cannot be 
used for some important purposes. This will be illustrated in Section 13.4. 


An important element in many electronic devices is a unit-delay machine, which produces as 
output the input string delayed by a specified amount of time. How can a finite-state machine 
be constructed that delays an input string by one unit of time, that is, produces as output the bit 
string Ox,x, ...x,_, given the input bit string x,x, ....x,? 


Solution: A delay machine can be constructed that has two possible inputs, namely, 0 and 1. The 
machine must have a start state sọ. Because the machine has to remember whether the previous 
input was a 0 or a 1, two other states s, and s, are needed, where the machine is in state s, if the 
previous input was 1 and in state s, if the previous input was 0. An output of 0 is produced for 
the initial transition from sy. Each transition from sı gives an output of 1, and each transition 
from s, gives an output of 0. The output corresponding to the input of a string x, ...x, is the 
string that begins with 0, followed by x,, followed by x), ..., ending with x,_,. The state diagram 
for this machine is shown in Figure 4. < 


Produce a finite-state machine that adds two positive integers using their binary expansions. 


Solution: When (x, .-. X1Xọ) and (Y, ... Y1Yo) are added, the following procedure (as described 
in Section 4.2) is followed. First, the bits xọ and yọ are added, producing a sum bit zy and a carry 
bit cg. This carry bit is either 0 or 1. Then, the bits x, and y; are added, together with the carry 
Cp. This gives a sum bit z, and a carry bit c}. This procedure is continued until the nth stage, 
where x,,, y,, and the previous carry c,_, are added to produce the sum bit z, and the carry bit 
C,» Which is equal to the sum bit z,,,;. 

A finite-state machine to carry out this addition can be constructed using just two states. 
For simplicity we assume that both the initial bits x,, and y, are 0 (otherwise we have to make 
special arrangements concerning the sum bit z,,,,). The start state sọ is used to remember that 
the previous carry is O (or for the addition of the rightmost bits). The other state, s,, is used to 
remember that the previous carry is 1. 

Because the inputs to the machine are pairs of bits, there are four possible inputs. We repre- 
sent these possibilities by 00 (when both bits are 0), 01 (when the first bit is 0 and the second is 
1), 10 (when the first bit is 1 and the second is 0), and 11 (when both bits are 1). The transitions 


TABLE 4 


EXAMPLE 7 


Definition 2 
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00, 0 


10,1 10,0 


FIGURE 4 A unit-delay machine. FIGURE 5 A finite-state machine for 
addition. 


and the outputs are constructed from the sum of the two bits represented by the input and the 
carry represented by the state. For instance, when the machine is in state s, and receives 01 as 
input, the next state is s} and the output is 0, because the sum that arises is 0+ 1+ 1 = (10). 
The state diagram for this machine is shown in Figure 5. < 


In a certain coding scheme, when three consecutive 1s appear in a message, the receiver of the 
message knows that there has been a transmission error. Construct a finite-state machine that 
gives a | as its current output bit if and only if the last three bits received are all 1s. 


Solution: Three states are needed in this machine. The start state sy remembers that the previous 
input value, if it exists, was not a 1. The state s} remembers that the previous input was a 1, but 
the input before the previous input, if it exists, was not a 1. The state s, remembers that the 
previous two inputs were 1s. 

An input of 1 takes sọ to s,, because now a 1, and not two consecutive 1s, has been read; it 
takes s, to sy, because now two consecutive 1s have been read; and it takes s, to itself, because 
at least two consecutive 1s have been read. An input of 0 takes every state to sọ, because this 
breaks up any string of consecutive 1s. The output for the transition from s, to itself when a 1 
is read is 1, because this combination of input and state shows that three consecutive 1s have 
been read. All other outputs are 0. The state diagram of this machine is shown in Figure 6. < 


FIGURE 6 A finite-state machine that gives an output of 1 
if and only if the input string read so far ends with 111. 


The final output bit of the finite-state machine we constructed in Example 7 is 1 if and only if 
the input string ends with 111. Because of this, we say that this finite-state machine recognizes 
the set of bit strings that end with 111. This leads us to Definition 2. 


Let M = (S, 1, O, f, 8, Sọ) be a finite-state machine and L C 1*. We say that M recognizes (or 
accepts) L if an input string x belongs to L if and only if the last output bit produced by M 
when given x as input is a 1. 
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Exercises 


1. Draw the state diagrams for the finite-state machines with a) 
these state tables. 


TYPES OF FINITE-STATE MACHINES Many different kinds of finite-state machines have 
been developed to model computing machines. In this section we have given a definition of 
one type of finite-state machine. In the type of machine introduced in this section, outputs cor- 
respond to transitions between states. Machines of this type are known as Mealy machines, 
because they were first studied by G. H. Mealy in 1955. There is another important type of 
finite-state machine with output, where the output is determined only by the state. This type of 
finite-state machine is known as a Moore machine, because E. F. Moore introduced this type 
of machine in 1956. Moore machines are considered in a sequence of exercises. 

In Example 7 we showed how a Mealy machine can be used for language recognition. 
However, another type of finite-state machine, giving no output, is usually used for this purpose. 
Finite-state machines with no output, also known as finite-state automata, have a set of final 
states and recognize a string if and only if it takes the start state to a final state. We will study 
this type of finite-state machine in Section 13.3. 


Start 


b) 


Start 


c) 


Start 


3. Find the output generated from the input string 01110 for 
the finite-state machine with the state table in 


a) Exercise 1(a). 
b) Exercise 1(b). 
c) Exercise 1(c). 
4. Find the output generated from the input string 10001 for 
the finite-state machine with the state diagram in 
a) Exercise 2(a). 


2. Give the state tables for the finite-state machines with b) Exercise 2(b). 


these state diagrams. 


c) Exercise 2(c). 


10. 


11. 


12. 


13. 


14. 


15. 


. Find the output for each of these input strings when given 


as input to the finite-state machine in Example 2. 
a) 0111 b) 11011011 c) 01010101010 


. Find the output for each of these input strings when given 


as input to the finite-state machine in Example 3. 
a) 0000 b) 101010 c) 11011100010 


. Construct a finite-state machine that models an old- 


fashioned soda machine that accepts nickels, dimes, and 
quarters. The soda machine accepts change until 35 cents 
has been put in. It gives change back for any amount 
greater than 35 cents. Then the customer can push but- 
tons to receive either a cola, a root beer, or a ginger ale. 


. Construct a finite-state machine that models a newspa- 


per vending machine that has a door that can be opened 
only after either three dimes (and any number of other 
coins) or a quarter and a nickel (and any number of other 
coins) have been inserted. Once the door can be opened, 
the customer opens it and takes a paper, closing the door. 
No change is ever returned no matter how much extra 
money has been inserted. The next customer starts with 
no credit. 


. Construct a finite-state machine that delays an input 


string two bits, giving 00 as the first two bits of output. 
Construct a finite-state machine that changes every other 
bit, starting with the second bit, of an input string, and 
leaves the other bits unchanged. 

Construct a finite-state machine for the log-on procedure 
for a computer, where the user logs on by entering a user 
identification number (a string of digits), which is con- 
sidered to be a single input, and then a password (a string 
of characters), which is considered to be a single input. 
If the password is incorrect, the user is asked for the user 
identification number again. 

Construct a finite-state machine for a combination lock 
that contains numbers 1 through 40 and that opens only 
when the correct combination, 10 right, 8 second left, 
37 right, is entered. Each input is a triple consisting of 
a number, the direction of the turn, and the number of 
times the lock is turned in that direction. 

Construct a finite-state machine for a toll machine that 
opens a gate after 25 cents, in nickels, dimes, or quar- 
ters, has been deposited. No change is given for overpay- 
ment, and no credit is given to the next driver when more 
than 25 cents has been deposited. 

Construct a finite-state machine for entering a security 
code into an automatic teller machine (ATM) that imple- 
ments these rules: A user enters a string of four digits, 
one digit at a time. If the user enters the correct four dig- 
its of the password, the ATM displays a welcome screen. 
When the user enters an incorrect string of four digits, the 
ATM displays a screen that informs the user that an incor- 
rect password was entered. If a user enters the incorrect 
password three times, the account is locked. 

Construct a finite-state machine for a restricted telephone 
switching system that implements these rules. Only calls 
to the telephone numbers 0, 911, and the digit 1 fol- 
lowed by 10-digit telephone numbers that begin with 212, 


16. 


17. 


18. 


19. 


13.2 Finite-State Machines with Output 903 


800, 866, 877, and 888 are sent to the network. All other 
strings of digits are blocked by the system and the user 
hears an error message. 

Construct a finite-state machine that gives an output of 
1 if the number of input symbols read so far is divisible 
by 3 and an output of 0 otherwise. 


Construct a finite-state machine that determines whether 
the input string has a 1 in the last position and a 0 in the 
third to the last position read so far. 

Construct a finite-state machine that determines whether 
the input string read so far ends in at least five consecu- 
tive ls. 


Construct a finite-state machine that determines whether 
the word computer has been read as the last eight char- 
acters in the input read so far, where the input can be any 
string of English letters. 


A Moore machine M = (S, 1, O, f, g, Sọ) consists of a finite set 
of states, an input alphabet 7, an output alphabet O, a transition 
function f that assigns a next state to every pair of a state and 
an input, an output function g that assigns an output to every 
state, and a starting state sọ. A Moore machine can be repre- 
sented either by a table listing the transitions for each pair of 
state and input and the outputs for each state, or by a state di- 
agram that displays the states, the transitions between states, 
and the output for each state. In the diagram, transitions are 
indicated with arrows labeled with the input, and the outputs 
are shown next to the states. 


20. 


21. 


Construct the state diagram for the Moore machine with 
this state table. 


Construct the state table for the Moore machine with 
the state diagram shown here. Each input string to a 
Moore machine M produces an output string. In particu- 
lar, the output corresponding to the input string a, da) ... ay 
is the string g(sp)g(s,) ... g(s,), where s; = f(s;_1, a;) for 
PS Deak. 


Start 
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23. 
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Find the output string generated by the Moore machine 
in Exercise 20 with each of these input strings. 


a) 0101 b) 111111 c) 11101110111 


Find the output string generated by the Moore ma- 
chine in Exercise 21 with each of the input strings in 
Exercise 22. 

Construct a Moore machine that gives an output of 1 
whenever the number of symbols in the input string 


25. 


read so far is divisible by 4 and an output of 0 
otherwise. 


Construct a Moore machine that determines whether an 
input string contains an even or odd number of 1s. The 
machine should give 1 as output if an even number of 1s 
are in the string and 0 as output if an odd number of 1s 
are in the string. 


Finite-State Machines with No Output 


Links » 


Definition 1 


EXAMPLE 1 


EXAMPLE 2 


13.3.1 Introduction 


One of the most important applications of finite-state machines is in language recognition. This 
application plays a fundamental role in the design and construction of compilers for program- 
ming languages. In Section 13.2 we showed that a finite-state machine with output can be used 
to recognize a language, by giving an output of 1 when a string from the language has been read 
and a 0 otherwise. However, there are other types of finite-state machines that are specially de- 
signed for recognizing languages. Instead of producing output, these machines have final states. 
A string is recognized if and only if it takes the starting state to one of these final states. 


13.3.2 Set of Strings 


Before discussing finite-state machines with no output, we will introduce some important back- 
ground material on sets of strings. The operations that will be defined here will be used exten- 
sively in our discussion of language recognition by finite-state machines. 


Suppose that A and B are subsets of V*, where V is a vocabulary. The concatenation of A 
and B, denoted by AB, is the set of all strings of the form xy, where x is a string in A and y is 
a string in B. 


Let A = {0, 11} and B = {1, 10, 110}. Find AB and BA. 


Solution: The set AB contains every concatenation of a string in A and a string in B. Hence, 
AB = {01, 010, 0110, 111, 1110, 11110}. The set BA contains every concatenation of a string in 
B and a string in A. Hence, BA = {10, 111, 100, 1011, 1100, 11011}. < 


Note that it is not necessarily the case that AB = BA when A and B are subsets of V*, where 
V is an alphabet, as Example 1 illustrates. 

From the definition of the concatenation of two sets of strings, we can define A”, for n = 
0, 1, 2, .... This is done recursively by specifying that 


A? = {A}, 
A™1=A"A forn=0,1,2,.... 


Let A = {1,00}. Find A” for n = 0, 1, 2, and 3. 


Solution: We have A? = {A} and A! = A°A = {A}A = {1,00}. To find A? we take concate- 
nations of pairs of elements of A. This gives A= {11, 100, 001, 0000}. To find A? we 
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take concatenations of elements in A? and A; this gives A= {111, 1100, 1001, 10000, 
0011, 00100, 00001, 000000}. < 


Definition 2 Suppose that A is a subset of V*. Then the Kleene closure of A, denoted by A*, is the set 
consisting of concatenations of arbitrarily many strings from A. That is, A* = (Jo AK., 


EXAMPLE 3 What are the Kleene closures of the sets A = {0}, B = {0, 1}, and C = {11}? 


Solution: The Kleene closure of A is the concatenation of the string 0 with itself an arbitrary 
finite number of times. Hence, A* = {0” | n = 0, 1, 2, ... }. The Kleene closure of B is the con- 
catenation of an arbitrary number of strings, where each string is either 0 or 1. This is the set 
of all strings over the alphabet V = {0, 1}. That is, B* = V*. Finally, the Kleene closurebreak 
of C is the concatenation of the string 11 with itself an arbitrary number of times. Hence, C* is 
the set of strings consisting of an even number of 1s. That is, C* = {177 |n=0,1,2,...}. < 


13.3.3 Finite-State Automata 


We will now give a definition of a finite-state machine with no output. Such machines are also 
called finite-state automata, and that is the terminology we will use for them here. (Note: The 
singular of automata is automaton.) These machines differ from the finite-state machines studied 
in Section 13.2 in that they do not produce output, but they do have a set of final states. As we 
will see, they recognize strings that take the starting state to a final state. 


Definition 3 A finite-state automaton M = (S, I, f, Sọ F) consists of a finite set S of states, a finite input 
alphabet I, a transition function f that assigns a next state to every pair of state and input 
(so that f : SX J —> S), an initial or start state sy, and a subset F of S consisting of final (or 
accepting states). 


We can represent finite-state automata using either state tables or state diagrams. Final states 
are indicated in state diagrams by using double circles. 


EXAMPLE 4 Construct the state diagram for the finite-state automaton M = (S, I, f, so, F), where S = 
{so Sp So, 83}, 1 = {0, 1}, F = {5o 53}, and the transition function f is given in Table 1. 


Solution: The state diagram is shown in Figure 1. Note that because both the inputs O and 1 
take s, to sy, we write 0,1 over the edge from s, to sọ. < 


Links » 


STEPHEN COLE KLEENE (1909-1994) Stephen Kleene was born in Hartford, Connecticut. His mother, Alice Lena Cole, was 
a poet, and his father, Gustav Adolph Kleene, was an economics professor. Kleene attended Amherst College and received his Ph.D. 
from Princeton in 1934, where he studied under the famous logician Alonzo Church. Kleene joined the faculty of the University of 
Wisconsin in 1935, where he remained except for several leaves, including stays at the Institute for Advanced Study in Princeton. 
During World War II he was a navigation instructor at the Naval Reserve’s Midshipmen’s School and later served as the director of 
the Naval Research Laboratory. Kleene made significant contributions to the theory of recursive functions, investigating questions 
of computability and decidability, and proved one of the central results of automata theory. He served as the Acting Director of the 
Mathematics Research Center and as Dean of the College of Letters and Sciences at the University of Wisconsin. Kleene was a 
student of natural history. He discovered a previously undescribed variety of butterfly that is named after him. He was an avid hiker 
and climber. Kleene was also noted as a talented teller of anecdotes, using a powerful voice that could be heard several offices away. 
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TABLE 1 


Definition 4 


EXAMPLE 5 


FIGURE 1 The state diagram for a 
finite-state automaton. 


EXTENDING THE TRANSITION FUNCTION The transition function f of a finite-state ma- 
chine M = (S, I, f Sọ F) can be extended so that it is defined for all pairs of states and strings; 
that is, f can be extended to a function f : S x I* > S. Let x = x,x, ... x, be a string in /*. Then 
f(s x) is the state obtained by using each successive symbol of x, from left to right, as in- 
put, starting with state s}. From s, we go on to state s, = f(s), x,), then to state s; = f (s2, x2), 
and so on, with f(s,, x) = f (Sp x,). Formally, we can define this extended transition function f 
recursively for the deterministic finite-state machine M = (S, I, f Sọ F) by 


(i) f(s, A) = s for every state s € S; and 
(ii) f(s, xa) = f (f(s, x), a) for alls E€ S, x E I*, anda E I. 


We can use structural induction and this recursive definition to prove properties of this extended 
transition function. For example, in Exercise 15 we ask you to prove that 


f(s, xy) = f(s, x), y) 


for every state s E€ S and strings x € /* and y € 1*. 


13.3.4 Language Recognition by Finite-State Machines 


Next, we define some terms that are used when studying the recognition by finite-state automata 
of certain sets of strings. 


A string x is said to be recognized or accepted by the machine M = (S, I, f, Sọ F) if it takes 
the initial state sọ to a final state, that is, f(sọ, x) is a state in F. The language recognized or 
accepted by the machine M, denoted by L(M), is the set of all strings that are recognized by 
M. Two finite-state automata are called equivalent if they recognize the same language. 


In Example 5 we will find the languages recognized by several finite-state automata. 


Determine the languages recognized by the finite-state automata M,, M,, and M; in Figure 2. 


Solution: The only final state of M, is sọ. The strings that take sọ to itself are those consisting 
of zero or more consecutive ls. Hence, L(M,) = {1" |n=0, 1, 2,...}. 

The only final state of M, is s,. The only strings that take sọ to s, are 1 and 01. Hence, 
L(M,) = {1,01}. 

The final states of M; are sọ and s43. The only strings that take sọ to itself are A, 0, 00, 000, ..., 
that is, any string of zero or more consecutive Os. The only strings that take sọ to s} are 
a string of zero or more consecutive Os, followed by 10, followed by any string. Hence, 
L(M,) = {0", 0" 10x | n = 0, 1, 2, ..., and x is any string}. < 


EXAMPLE 6 


Extra 
Examples 


» 
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FIGURE 2 Some finite-state automata. 


DESIGNING FINITE-STATE AUTOMATA We can often construct a finite-state automaton 
that recognizes a given set of strings by carefully adding states and transitions and determining 
which of these states should be final states. When appropriate we include states that can keep 
track of some of the properties of the input string, providing the finite-state automaton with 
limited memory. Examples 6 and 7 illustrate some of the techniques that can be used to construct 
finite-state automata that recognize particular types of sets of strings. 


Construct deterministic finite-state automata that recognize each of these languages. 


(a) the set of bit strings that begin with two Os 

(b) the set of bit strings that contain two consecutive Os 

(c) the set of bit strings that do not contain two consecutive Os 
(d) the set of bit strings that end with two Os 

(e) the set of bit strings that contain at least two Os 


Solution: (a) Our goal is to construct a deterministic finite-state automaton that recog- 
nizes the set of bit strings that begin with two Os. Besides the start state sọ, we include a 
nonfinal state s}; we move to s, from sọ if the first bit is a 0. Next, we add a final state sy, 
which we move to from s, if the second bit is a 0. When we have reached s, we know that the 
first two input bits are both Os, so we stay in the state s, no matter what the succeeding bits (if 
any) are. We move to a nonfinal state s} from sọ if the first bit is a 1 and from s, if the second 
bit is a 1. The reader should verify that the finite-state automaton in Figure 3(a) recognizes the 
set of bit strings that begin with two Os. 
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FIGURE 3 Deterministic finite-state automata recognizing the languages in Example 6. 


(b) Our goal is to construct a deterministic finite-state automaton that recognizes the set of 
bit strings that contain two consecutive Os. Besides the start state sọ, we include a nonfinal 
state s}, which tells us that the last input bit seen is a 0, but either the bit before it was a 1, or 
this bit was the initial bit of the string. We include a final state s, that we move to from sı when 
the next input bit after a 0 is also a 0. If a 1 follows a 0 in the string (before we encounter two 
consecutive Os), we return to sọ and begin looking for consecutive Os all over again. The reader 
should verify that the finite-state automaton in Figure 3(b) recognizes the set of bit strings that 
contain two consecutive Os. 


(c) Our goal is to construct a deterministic finite-state automaton that recognizes the set of 
bit strings that do not contain two consecutive Os. Besides the start state sy, which should 
be a final state, we include a final state s}, which we move to from sọ when 0 is the first in- 
put bit. When an input bit is a 1, we return to, or stay in, state sọ. We add a state s,, which 
we move to from s, when the input bit is a 0. Reaching s, tells us that we have seen two 
consecutive Os as input bits. We stay in state s, once we have reached it; this state is not fi- 
nal. The reader should verify that the finite-state automaton in Figure 3(c) recognizes the set of 
bit strings that do not contain two consecutive Os. [The astute reader will notice the relationship 
between the finite-state automaton constructed here and the one constructed in part (b). See 
Exercise 39.] 


(d) Our goal is to construct a deterministic finite-state automaton that recognizes the set of bit 
strings that end with two Os. Besides the start state sọ, we include a nonfinal state s}, which 
we move to if the first bit is 0. We include a final state s,, which we move to from s, if the 
next input bit after a 0 is also a 0. If an input of 0 follows a previous 0, we stay in state s, 
because the last two input bits are still Os. Once we are in state s, an input bit of 1 sends us back 
to sg, and we begin looking for consecutive Os all over again. We also return to sọ if the next 
input is a 1 when we are in state s}. The reader should verify that the finite-state automaton in 
Figure 3(d) recognizes the set of bit strings that end with two Os. 


(e) Our goal is to construct a deterministic finite-state automaton that recognizes the set of bit 
strings that contain two Os. Besides the start state, we include a state s}, which is not final; we 
stay in sọ until an input bit is a 0 and we move to s,; when we encounter the first 0 bit in the input. 
We add a final state s, which we move to from sı once we encounter a second 0 bit. Whenever 
we encounter a l as input, we stay in the current state. Once we have reached sj, we remain 


EXAMPLE 7 
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there. Here, s; and s, are used to tell us that we have already seen one or two Os in the input 
string so far, respectively. The reader should verify that the finite-state automaton in Figure 3(e) 
recognizes the set of bit strings that contain two Os. < 


Construct a deterministic finite-state automaton that recognizes the set of bit strings that contain 
an odd number of 1s and that end with at least two consecutive Os. 


Solution: We can build a deterministic finite-state automaton that recognizes the specified set 
by including states that keep track of both the parity of the number of 1 bits and whether we 
have seen no, one, or at least two Os at the end of the input string. 

The start state sy can be used to tell us that the input read so far contains an even number 
of 1s and ends with no Os (that is, is empty or ends with a 1). Besides the start state, we include 
five more states. We move to states 5), S2, 53, 54, and s,, respectively, when the input string read 
so far contains an even number of Is and ends with one 0; when it contains an even number 
of 1s and ends with at least two Os; when it contains an odd number of 1s and ends with no Os; 
when it contains an odd number of 1s and ends with one 0; and when it contains an odd number 
of 1s and ends with two Os. The state s; is a final state. 

The reader should verify that the finite-state automaton in Figure 4 recognizes the set of bit 
strings that contain an odd number of 1s and end with at least two consecutive Os. < 


Start 
0 
Number of even even even odd odd odd 
Is 
Number of 0 1 >2 0 1 >2 
Os at end 


FIGURE 4 A deterministic finite-state automaton recognizing the set of bit strings containing an odd 
number of 1s and ending with at least two Os. 


EXAMPLE 8 


EQUIVALENT FINITE-STATE AUTOMATA In Definition 4 we specified that two finite- 
state automata are equivalent if they recognize the same language. Example 8 provides an ex- 
ample of two equivalent deterministic finite-state machines. 


Show that the two finite-state automata Mọ and M, shown in Figure 5 are equivalent. 


Solution: For a string x to be recognized by Mp, x must take us from sọ to the final state s, or 
the final state s4. The only string that takes us from sọ to s, is the string 1. The strings that take 
us from sọ to s4 are those strings that begin with a 0, which takes us from sọ to sy, followed by 
zero or more additional Os, which keep the machine in state s, followed by a 1, which takes us 
from state s, to the final state s4. All other strings take us from sọ to a state that is not final. (We 
leave it to the reader to fill in the details.) We conclude that L(Mọ) is the set of strings of zero 
or more 0 bits followed by a final 1. 
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0 
Start © 0,1 
i 0,1 


M; 


FIGURES M, and M, are equivalent finite-state automata. 


For a string x to be recognized by M,, x must take us from sọ to the final state s4. So, for x to 
be recognized, it must begin with some number of Os, which leave us in state sy, followed by a 
1, which takes us to the final state s,. A string of all zeros is not recognized because it leaves us 
in state sọ, which is not final. All strings that contain a 0 after 1 are not recognized because they 
take us to state s,, which is not final. It follows that L(M,) is the same as L(M)). We conclude 
that My and M, are equivalent. 

Note that the finite-state machine M, only has three states. No finite state machine with 
fewer than three states can be used to recognize the set of all strings of zero or more 0 bits 
followed by a 1 (see Exercise 37). < 


As Example 8 shows, a finite-state automaton may have more states than one equivalent 
to it. In fact, algorithms used to construct finite-state automata to recognize certain languages 
may have many more states than necessary. Using unnecessarily large finite-state machines to 
recognize languages can make both hardware and software applications inefficient and costly. 
This problem arises when finite-state automata are used in compilers, which translate computer 
programs to a language a computer can understand (object code). 

Exercises 58-61 develop a procedure that constructs a finite-state automaton with the fewest 
states possible among all finite-state automata equivalent to a given finite-state automaton. This 
procedure is known as machine minimization. The minimization procedure described in these 
exercises reduces the number of states by replacing states with equivalence classes of states with 
respect to an equivalence relation in which two states are equivalent if every input string either 
sends both states to a final state or sends both to a state that is not final. Before the minimization 
procedure begins, all states that cannot be reached from the start state using any input string are 
first removed; removing these does not change the language recognized. 


13.3.5 Nondeterministic Finite-State Automata 
The finite-state automata discussed so far are deterministic, because for each pair of state and 


input value there is a unique next state given by the transition function. There is another im- 
portant type of finite-state automaton in which there may be several possible next states for 
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each pair of input value and state. Such machines are called nondeterministic. Nondetermin- 
istic finite-state automata are important in determining which languages can be recognized by 
a finite-state automaton. 


Definition 5 A nondeterministic finite-state automaton M = (S, I, f, Sọ, F) consists of a set S of states, an 
Links > input alphabet /, a transition function f that assigns a set of states to each pair of state and 
input (so that f : S x J > P(S)), a starting state sy, and a subset F of S consisting of the final 

states. 


We can represent nondeterministic finite-state automata using state tables or state diagrams. 
When we use a state table, for each pair of state and input value we give a list of possible next 
states. In the state diagram, we include an edge from each state to all possible next states, labeling 
edges with the input or inputs that lead to this transition. 


EXAMPLE 9 Find the state diagram for the nondeterministic finite-state automaton with the state table shown 
in Table 2. The final states are s, and sz. 


Solution: The state diagram for this automaton is shown in Figure 6. < 


GRACE BREWSTER MURRAY HOPPER (1906-1992) Grace Hopper, born in New York City, dis- 
| played an intense curiosity as a child with how things worked. At the age of seven, she disassem- 
) >) bled alarm clocks to discover their mechanisms. She inherited her love of mathematics from her mother, 
“| who received special permission to study geometry (but not algebra and trigonometry) at a time when 

women were actively discouraged from such study. Hopper was inspired by her father, a successful in- 
surance broker, who had lost his legs from circulatory problems. He told his children they could do any- 
thing if they put their minds to it. He inspired Hopper to pursue higher education and not conform to 
the usual roles for females. Her parents made sure that she had an excellent education; she attended 
©Bettmann/Getty Images private schools for girls in New York. Hopper entered Vassar College in 1924, where she majored in 

mathematics and physics; she graduated in 1928. She received a masters degree in mathematics from 

Yale University in 1930. In 1930 she also married an English instructor at the New York School of Commerce; she later divorced 
and did not have children. Hopper was a mathematics professor at Vassar from 1931 until 1943, earning a Ph.D. from Yale in 1934. 

After the attack on Pearl Harbor, Hopper, coming from a family with strong military traditions, decided to leave her academic 
position and join the Navy WAVES. To enlist, she needed special permission to leave her strategic position as a mathematics professor, 
as well as a waiver for weighing too little. In December 1943, she was sworn into the Navy Reserve and trained at the Midshipman’s 
School for Women. Hopper was assigned to work at the Naval Ordnance Laboratory at Harvard University. She wrote programs for 
the world’s first large-scale automatically sequenced digital computer, which was used to help aim Navy artillery in varying weather. 
Hopper has been credited with coining the term “bug” to refer to a hardware glitch, but it was used at Harvard prior to her arrival 
there. However, it is true that Hopper and her programming team found a moth in one of the relays in the computer hardware that 
shut the system down. This famous moth was pasted into a lab book. In the 1950s Hopper coined the term “debug” for the process 
of removing programming errors. 

In 1946, when the Navy told her that she was too old for active service, Hopper chose to remain at Harvard as a civilian research 
fellow. In 1949 she left Harvard to join the Eckert-Mauchly Computer Corporation, where she helped develop UNIVAC, the first 
commercial computer. Hopper remained with this company when it was taken over by Remington Rand and when Remington Rand 
merged with the Sperry Corporation. She was a visionary for the potential power of computers; she understood that computers 
would become widely used if tools that were both programmer-friendly and application-friendly could be developed. In particular, 
she believed that computer programs could be written in English, rather than using machine instructions. To help achieve this goal, 
she developed the first compiler. She published the first research paper on compilers in 1952. Hopper is also known as the mother 
of the computer language COBOL; members of Hopper’s staff helped to frame the basic language design for COBOL using their 
earlier work as a basis. 

In 1966, Hopper retired from the Navy Reserve. However, only seven months later, the Navy recalled her from retirement to 
help standardize high-level naval computer languages. In 1983 she was promoted to the rank of Commodore by special Presidential 
appointment, and in 1985 she was elevated to the rank of Rear Admiral. Her retirement from the Navy, at the age of 80, was held on 
the USS Constitution. 
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TABLE 2 


FIGURE 6 The nondeterministic 
finite-state automaton with state 
table given in Table 2 


EXAMPLE 10 Find the state table for the nondeterministic finite-state automaton with the state diagram shown 


EXAMPLE 11 


in Figure 7. 


Solution: The state table is given as Table 3. < 


What does it mean for a nondeterministic finite-state automaton to recognize a string 
xX = XX3 ...x,? The first input symbol x, takes the starting state sọ to a set S, of states. The 
next input symbol x, takes each of the states in S, to a set of states. Let S, be the union of 
these sets. We continue this process, including, at each stage, all states obtained using a state 
obtained at the previous stage and the current input symbol. We recognize, or accept, the string 
x if there is a final state in the set of all states that can be obtained from sọ using x. The language 
recognized by a nondeterministic finite-state automaton is the set of all strings recognized by 
this automaton. 


Find the language recognized by the nondeterministic finite-state automaton shown in Figure 7. 


Solution: Because sọ is a final state, and there is a transition from sọ to itself when O is the 
input, the machine recognizes all strings consisting of zero or more consecutive Os. Furthermore, 
because s4 is a final state, any string that has s4 in the set of states that can be reached from sy 
with this input string is recognized. The only such strings are strings consisting of zero or more 
consecutive Os followed by 01 or 11. Because sọ and s4 are the only final states, the language 
recognized by the machine is {0”, 0"01, 0"11 | n > 0}. < 


TABLE —a 


Start 


FIGURE 7 A nondeterministic finite-state 
automaton. 
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One important fact is that a language recognized by a nondeterministic finite-state automa- 
ton is also recognized by a deterministic finite-state automaton. We will take advantage of this 
fact in Section 13.4 when we will determine which languages are recognized by finite-state 
automata. 


If the language L is recognized by a nondeterministic finite-state automaton Mp, then L is 
also recognized by a deterministic finite-state automaton M}. 


Proof: We will describe how to construct the deterministic finite-state automaton M, that rec- 
ognizes L from Mo, the nondeterministic finite-state automaton that recognizes this language. 
Each state in M, will be made up of a set of states in My. The start symbol of M, is {sọ}, which 
is the set containing the start state of Mo. The input set of M, is the same as the input set of Mp. 

Given a state {5;,5;,,...,5;,} Of M,, the input symbol x takes this state to the union 
of the sets of next states for the elements of this set, that is, the union of the sets F(S;, x) 
Fisip x), ving d (Sips x). The states of M, are all the subsets of S, the set of states of Mp, that are 
obtained in this way starting with sọ. (There are as many as 2” states in the deterministic ma- 
chine, where n is the number of states in the nondeterministic machine, because all subsets may 
occur as states, including the empty set, although usually far fewer states occur.) The final states 
of M, are those sets that contain a final state of Mọ. 

Suppose that an input string is recognized by Mo. Then one of the states that can be reached 
from sọ using this input string is a final state (the reader should provide an inductive proof of 
this). This means that in M,, this input string leads from {sọ} to a set of states of Mọ that contains 
a final state. This subset is a final state of M}, so this string is also recognized by M}. Also, an 
input string not recognized by Mọ does not lead to any final states in Mọ. (The reader should 
provide the details that prove this statement.) Consequently, this input string does not lead from 
{so} to a final state in M}. < 


Find a deterministic finite-state automaton that recognizes the same language as the nondeter- 
ministic finite-state automaton in Example 10. 


Solution: The deterministic automaton shown in Figure 8 is constructed from the nondetermin- 
istic automaton in Example 10. The states of this deterministic automaton are subsets of the set 
of all states of the nondeterministic machine. The next state of a subset under an input symbol 
is the subset containing the next states in the nondeterministic machine of all elements in this 
subset. For instance, on input of 0, {sq} goes to {So S2}, because sọ has transitions to itself and 
to sy in the nondeterministic machine; the set {5 , s2} goes to {s,,.54} on input of 1, because 


FIGURE 8 A deterministic automaton equivalent to the 
nondeterministic automaton in Example 10. 
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So goes just to s, and s, goes just to s4 on input of 1 in the nondeterministic machine; and the 
set {5,, $4} goes to {s3 } on input of 0, because s, and s4 both go to just s, on input of 0 in the 
deterministic machine. All subsets that are obtained in this way are included in the deterministic 
finite-state machine. Note that the empty set is one of the states of this machine, because it is 
the subset containing all the next states of {53} on input of 1. The start state is {sọ}, and the set 
of final states are all those that include sọ or s4. < 


Exercises 


. Let A = {0,11} and B = {00, 01}. Find each of these 


sets. 


a) AB b) BA c) A? d) B? 


2. Show that if A is a set of strings, then AØ = ØA = Ø. 


10. 


11. 


12. 


13. 


. Find all pairs of sets of strings A and B for which AB = 


{10, 111, 1010, 1000, 10111, 101000}. 


. Show that these equalities hold. 


a) {A}* = {4} 
b) (A*)* = A* for every set of strings A 


. Describe the elements of the set A* for these values of A. 


a) {10} b) {111} c) {0,01} d) {1, 101} 


. Let V be an alphabet, and let A and B be subsets of V*. 


Show that |AB| < |A||B]. 


. Let V be an alphabet, and let A and B be subsets of V* 


with A C B. Show that A* C B*. 


. Suppose that A is a subset of V*, where V is an alphabet. 


Prove or disprove each of these statements. 


a) ACA? b) if A =A’, then å E A 
c) A{A}=A d) (A*)* = A* 
e) A*A = A* f) |A"| = |A|” 
. Determine whether the string 11101 is in each of these 
sets. 
a) {0, 1}* b) {1}*{O}*{1}* 


c) {11} {0}*{01} da) {11}*{O1}* 

e) {111}*{O}*{1} f) {11,0} {00, 101} 
Determine whether the string 01001 is in each of these 
sets. 

a) {0, 1}* b) (O}*{10}{1}* 

c) {O10}*{O}* {1} d) {010,011} {00, 01} 

e) {00} {O}*{01} £) {01}*{01}* 

Determine whether each of these strings is recognized 
by the deterministic finite-state automaton in Figure 1. 
a) 111 b) 0011 c) 1010111 d) 011011011 
Determine whether each of these strings is recognized 
by the deterministic finite-state automaton in Figure 1. 
a) 010 b) 1101 c) 1111110 d) 010101010 
Determine whether all the strings in each of these sets are 
recognized by the deterministic finite-state automaton in 
Figure 1. 


a) {0}* 
d) {01}* 


b) {0} {0}* 
e) {O}*{1}* 


c) {1} {0}* 
f) {1} {0 1}* 


14. 


15. 


Show that if M = (S, I, f, so, F) is a deterministic finite- 
state automaton and f(s, x) = s for the state s € S and the 
input string x € /*, then f(s, x”) = s for every nonnega- 
tive integer n. (Here x” is the concatenation of n copies 
of the string x, defined recursively in Exercise 37 in 
Section 5.3.) 


Given a deterministic finite-state automaton M = 
(S, Lf, So, F), use structural induction and the recur- 
sive definition of the extended transition function f to 
prove that f(s, xy) =f(f(s, x), y) for all states s E S and 
all strings x € J* andy € 7*. 


In Exercises 16-22 find the language recognized by the given 
deterministic finite-state automaton. 


16. 


17. 


18. 


19. 


“OOO 
I 
1 
Start O 1 (®) 0 oon 
0 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings beginning with 01. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that end with 10. 


Construct a deterministic finite-state automaton that 
recognizes the set of all bit strings that contain the 
string 101. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that do not contain three 
consecutive Os. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that contain exactly 
three Os. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that contain at least 
three Os. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that contain three consec- 
utive ls. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that begin with 0 or 
with 11. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that begin and end 
with 11. 


32. 


33. 


34. 


35. 


36. 


37. 


38. 


39. 


40. 


41. 


42. 
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Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that contain an even num- 
ber of 1s. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that contain an odd num- 
ber of Os. 


Construct a deterministic finite-state automaton that rec- 
ognizes the set of all bit strings that contain an even num- 
ber of Os and an odd number of 1s. 


Construct a finite-state automaton that recognizes the set 
of bit strings consisting of a 0 followed by a string with 
an odd number of 1s. 


Construct a finite-state automaton with four states that 
recognizes the set of bit strings containing an even num- 
ber of 1s and an odd number of Os. 


Show that there is no finite-state automaton with two 
states that recognizes the set of all bit strings that have 
one or more 1 bits and end with a 0. 


Show that there is no finite-state automaton with three 
states that recognizes the set of bit strings containing an 
even number of 1s and an even number of Os. 


Explain how you can change the deterministic finite-state 
automaton M so that the changed automaton recognizes 
the set J* — L(M). 


Use Exercise 39 and finite-state automata constructed in 
Example 6 to find deterministic finite-state automata that 
recognize each of these sets. 


a) the set of bit strings that do not begin with two Os 
b) the set of bit strings that do not end with two Os 


c) the set of bit strings that contain at most one O (that 
is, that do not contain at least two Os) 


Use the procedure you described in Exercise 39 and the 
finite-state automata you constructed in Exercise 25 to 
find a deterministic finite-state automaton that recognizes 
the set of all bit strings that do not contain the string 101. 


Use the procedure you described in Exercise 39 and the 
finite-state automaton you constructed in Exercise 29 to 
find a deterministic finite-state automaton that recognizes 
the set of all bit strings that do not contain three consec- 
utive 1s. 


In Exercises 43—49 find the language recognized by the given 
nondeterministic finite-state automaton. 


Ga) 1 


43. 
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1,0 


50. Find a deterministic finite-state automaton that recog- 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 43. 

51. Find a deterministic finite-state automaton that recog- 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 44. 

52. Find a deterministic finite-state automaton that recog- 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 45. 

53. Find a deterministic finite-state automaton that recog- 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 46. 

54. Find a deterministic finite-state automaton that recog- 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 47. 


55. Find a deterministic finite-state automaton that recog- 
nizes each of these sets. 


a) {0} b) {1,00} 
c) {1"|n=2,3,4,...} 

56. Find a nondeterministic finite-state automaton that recog- 
nizes each of the languages in Exercise 55, and has fewer 


states, if possible, than the deterministic automaton you 
found in that exercise. 


*57. Show that there is no finite-state automaton that recog- 


nizes the set of bit strings containing an equal number 
of Os and 1s. 


In Exercises 58—62 we introduce a technique for constructing 
a deterministic finite-state machine equivalent to a given de- 
terministic finite-state machine with the least number of states 
possible. Suppose that M = (S, I, f, Sọ, F) is a finite-state au- 
tomaton and that k is a nonnegative integer. Let R, be the re- 
lation on the set S of states of M such that sR,t if and only if 
for every input string x with /(x) < k [where /(x) is the length 
of x, as usual], f(s, x) and f (t, x) are both final states or both not 
final states. Furthermore, let R, be the relation on the set of 
states of M such that sR,t if and only if for every input string 
x, regardless of length, f(s, x) and f(t, x) are both final states 
or both not final states. 


*58. a) Show that for every nonnegative integer k, R, is an 


equivalence relation on S. We say that two states s 
and t are k-equivalent if sR,t. 


b) Show that R, is an equivalence relation on S. We say 
that two states s and f are +-equivalent if sR,t. 


c) Show that if s and ¢ are two k-equivalent states of M, 
where k is a positive integer, then s and k are also 
(k — 1)-equivalent. 

d) Show that the equivalence classes of R, are a refine- 
ment of the equivalence classes of R,_, if k is a pos- 
itive integer. (The refinement of a partition of a set 
is defined in the preamble to Exercise 49 in Section 
9.5.) 


e) Show that if s and ż are k-equivalent for every non- 
negative integer k, then they are +-equivalent. 

f) Show that all states in a given R,,-equivalence class 
are final states or all are not final states. 


g) Show that if s and ¢ are R,,-equivalent, then f(s, a) and 
f(t a) are also R,,-equivalent for all a € J. 


* 59, Show that there is a nonnegative integer n such that the 


set of n-equivalence classes of states of M is the same as 
the set of (n + 1)-equivalence classes of states of M. Then 
show for this integer n, the set of n-equivalence classes 
of states of M equals the set of *-equivalence classes of 
states of M. 
The quotient automaton M of the deterministic finite-state 
automaton M = (S, L f, Sọ F) is the finite-state automaton 
(S, Lf, [soley F), where the set of states S is the set of 
-equivalence classes of S, the transition function f is defined 
by f(Ls] rR. ® = [f(S,@]p, for all states [s]p, of M and input 
symbols a € J, and F is the set consisting of R,,-equivalence 
classes of final states of M. 


* 60. a) 


b) 


c) 


** 61. a) 


b) 


Show that s and ż are 0-equivalent if and only if either 
both s and ¢ are final states or neither s nor t is a fi- 
nal state. Conclude that each final state of M, which 
is an R,-equivalence class, contains only final states 
of M. 

Show that if k is a positive integer, then s and f are k- 
equivalent if and only if s and ¢ are (k — 1)-equivalent 
and for every input symbola € I, f(s, a) and f (t, a) are 
(k — 1)-equivalent. Conclude that the transition func- 
tion f is well-defined. 

Describe a procedure that can be used to construct the 
quotient automaton of a finite-automaton M. 

Show that if M is a finite-state automaton, then the 
quotient automaton M recognizes the same language 
as M. 

Show that if M is a finite-state automaton with 
the property that for every state s of M there is a 
string x € J* such that f (sọ, x) = s, then the quotient 
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automaton M has the minimum number of states of 
any finite-state automaton equivalent to M. 


62. Answer these questions about the finite-state automaton 
M shown here. 


a) Find the k-equivalence classes of M for k = 0, 1, 2, 


and 3. Also, find the *-equivalence classes of M. 


b) Construct the quotient automaton M of M. 


Language Recognition 


Definition 1 


13.4.1 Introduction 


We have seen that finite-state automata can be used as language recognizers. What sets can be 
recognized by these machines? Although this seems like an extremely difficult problem, there 
is a simple characterization of the sets that can be recognized by finite state automata. This 
problem was first solved in 1956 by the American mathematician Stephen Kleene. He showed 
that there is a finite-state automaton that recognizes a set if and only if this set can be built 
up from the null set, the empty string, and singleton strings by taking concatenations, unions, 
and Kleene closures, in arbitrary order. Sets that can be built up in this way are called regular 
sets. Regular grammars were defined in Section 13.1. Because of the terminology used, it is 
not surprising that there is a connection between regular sets, which are the sets recognized by 
finite-state automata, and regular grammars. In particular, a set is regular if and only if it is 
generated by a regular grammar. 

Finally, there are sets that cannot be recognized by any finite-state automata. We will give 
an example of such a set. We will briefly discuss more powerful models of computation, such as 
pushdown automata and Turing machines, at the end of this section. The regular sets are those 
that can be formed using the operations of concatenation, union, and Kleene closure in arbitrary 
order, starting with the empty set, the set consisting of the empty string, and singleton sets. We 
will see that the regular sets are those that can be recognized using a finite-state automaton. To 
define regular sets we first need to define regular expressions. 


The regular expressions over a set I are defined recursively by: 


the symbol Ø is a regular expression; 

the symbol / is a regular expression; 

the symbol x is a regular expression whenever x € /; 

the symbols (AB), (A U B), and A* are regular expressions whenever A 
and B are regular expressions. 
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EXAMPLE 1 


EXAMPLE 2 


Each regular expression represents a set specified by these rules: 


Ø represents the empty set, that is, the set with no strings; 

A represents the set {A}, which is the set containing the empty string; 

x represents the set {x} containing the string with one symbol x; 

(AB) represents the concatenation of the sets represented by A and by B; 
(A U B) represents the union of the sets represented by A and by B; 

A* represents the Kleene closure of the set represented by A. 


Sets represented by regular expressions are called regular sets. Henceforth regular expressions 
will be used to describe regular sets, so when we refer to the regular set A, we will mean the 
regular set represented by the regular expression A. Note that we will leave out outer parentheses 
from regular expressions when they are not needed. 

Example | shows how regular expressions are used to specify regular sets. 


What are the strings in the regular sets specified by the regular expressions 10*, (10)*, 0 U 01, 
0(0 U 1)*, and (0*1)*? 


Solution: The regular sets represented by these expressions are given in Table 1, as the reader 
should verify. < 


Finding a regular expression that specifies a given set can be quite tricky, as Example 2 
illustrates. 


Find a regular expression that specifies each of these sets: 


(a) the set of bit strings with even length 
(b) the set of bit strings ending with a 0 and not containing 11 
(c) the set of bit strings containing an odd number of Os 


Solution: (a) To construct a regular expression for the set of bit strings with even length, we 
use the fact that such a string can be obtained by concatenating zero or more strings each 
consisting of two bits. The set of strings of two bits is specified by the regular expression 
(00 U01U10U11). Consequently, the set of strings with even length is specified by 
(00 u 01u 10u11)*. 


(b) A bit string ending with a O and not containing 11 must be the concatenation of one or 
more strings where each string is either a 0 or a 10. (To see this, note that such a bit string must 
consist of 0 bits or 1 bits each followed by a 0; the string cannot end with a single 1 because 
we know it ends with a 0.) It follows that the regular expression (0 U 10)* (0 U 10) specifies the 
set of bit strings that do not contain 11 and end with a 0. [Note that the set specified by (0 U 
10)* includes the empty string, which is not in this set, because the empty string does not end 
with a 0.] 


TABLE 1 


Expression Strings 


10* a 1 followed by any number of Os (including no zeros) 
(10)* any number of copies of 10 (including the null string) 
OU01 the string 0 or the string 01 


00 U 1)* any string beginning with 0 


(0*1)* any string not ending with 0 


THEOREM 1 
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(c) A bit string containing an odd number of Os must contain at least one 0, which tells us 
that it starts with zero or more 1s, followed by a 0, followed by zero or more 1s. That is, each 
such bit string begins with a string of the form 1/01‘ for nonnegative integers j and k. Because 
the bit string contains an odd number of Os, additional bits after this initial block can be split into 
blocks each starting with a 0 and containing one more 0. Each such block is of the form 01014, 
where p and q are nonnegative integers. Consequently, the regular expression 1*01*(01*01*)* 
specifies the set of bit strings with an odd number of Os. < 


13.4.2 Kleene’s Theorem 


In 1956 Kleene proved that regular sets are the sets that are recognized by a finite-state automa- 
ton. Consequently, this important result is called Kleene’s theorem. 


KLEENE’S THEOREM A set is regular if and only if it is recognized by a finite-state 
automaton. 


Kleene’s theorem is one of the central results in automata theory. We will prove the only if part 
of this theorem, namely, that every regular set is recognized by a finite-state automaton. The 
proof of the if part, that a set recognized by a finite-state automaton is regular, is left as an 
exercise for the reader. 


Proof: Recall that a regular set is defined in terms of regular expressions, which are defined 
recursively. We can prove that every regular set is recognized by a finite-state automaton if we 
can do the following things. 


1. Show that Ø is recognized by a finite-state automaton. 

. Show that {A} is recognized by a finite-state automaton. 

. Show that {a} is recognized by a finite-state automaton whenever a is a symbol in /. 
. Show that AB is recognized by a finite-state automaton whenever both A and B are. 


n AUN 


. Show that A U B is recognized by a finite-state automaton whenever both A and B are. 
6. Show that A* is recognized by a finite-state automaton whenever A is. 


We now consider each of these tasks. First, we show that Ø is recognized by a nondeterministic 
finite-state automaton. To do this, all we need is an automaton with no final states. Such an 
automaton is shown in Figure 1(a). 

Second, we show that {4 } is recognized by a finite-state automaton. To do this, all we need 
is an automaton that recognizes A, the null string, but not any other string. This can be done by 
making the start state sọ a final state and having no transitions, so that no other string takes so 
to a final state. The nondeterministic automaton in Figure 1(b) shows such a machine. 

Third, we show that {a} is recognized by a nondeterministic finite-state automaton. To do 
this, we can use a machine with a starting state sọ and a final state s}. We have a transition from 
So to sı when the input is a, and no other transitions. The only string recognized by this machine 
is a. This machine is shown in Figure 1(c). 


son —e(® Start sn —o( O) 


(a) (b) (c) 


FIGURE 1 Nondeterministic finite-state automata that recognize some basic sets. 
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Next, we show that AB and A U B can be recognized by finite-state automata if A and B 
are languages recognized by finite-state automata. Suppose that A is recognized by M, = 
(Sy, L fa Sa Fa) and B is recognized by Mz = (Sz, L fp, Sp Fp). 

We begin by constructing a finite-state machine My, = (Sip, L fap 9p Fap) that recognizes 
AB, the concatenation of A and B. We build such a machine by combining the machines for 
A and B in series, so a string in A takes the combined machine from s4, the start state of M4, 
to sz, the start state of Mp. A string in B should take the combined machine from sg to a final 
state of the combined machine. Consequently, we make the following construction. Let $4, be 
S, U Sp. [Note that we can assume that S4 and Sz are disjoint.] The starting state 5, is the same 
as s4. The set of final states, F4pg, is the set of final states of Mp with s4, included if and only if 
A E€ ANB. The transitions in M,, include all transitions in M, and in Mg, as well as some new 
transitions. For every transition in M, that leads to a final state, we form a transition in My, 
from the same state to sz, on the same input. In this way, a string in A takes Myp from 54p to Sp, 
and then a string in B takes sz to a final state of M,,. Moreover, for every transition from sz we 
form a transition in M,, from s,4, to the same state. Figure 2(a) contains an illustration of this 
construction. 


Transition to final state in M, produces a transition to sp. 


Transition from sg in Mz produces a transition from s4z = S4- 


Start state is 843 = 84, which is final if s4 and sz are final. Final states include all final states of Mp. 


(b) 


Start 


Final states are the final states in M, and Mp. 


Saug iS the new start state, which is final if s4 or sg is final. 


(c) 


Transitions from s, produce A transitions from s4 » and all final states of M4. 


sais the new start state, which is a final state. Final states include all final states in M,. 


FIGURE 2 Building automata to recognize concatenations, unions, and Kleene closures. 


EXAMPLE 3 
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We now construct a machine Myyp = (Saus L aue Saug aug) that recognizes A U B. This 
automaton can be constructed by combining M, and Mp in parallel, using a new start state 
that has the transitions that both s, and sg have. Let S4ug = S4 U Sg U {S4ug}, where sayz is a 
new state that is the start state of M,,,,. Let the set of final states Fy, be Fy U Fg U {Saug} if 
A E A U B, and F, U Fz otherwise. The transitions in M4. include all those in M, and in Mz. 
Also, for each transition from s4 to a state s on input i we include a transition from sS4ug to s on 
input 7, and for each transition from sz to a state s on input į we include a transition from s yup 
to s on input i. In this way, a string in A leads from 54. to a final state in the new machine, 
and a string in B leads from s4ņg to a final state in the new machine. Figure 2(b) illustrates the 
construction of Mjyp. 

Finally, we construct M ee = (Sy 1, f ae 54%, Fx), a machine that recognizes A*, the Kleene 
closure of A. Let S,» include all states in $4 and one additional state s,*, which is the starting 
state for the new machine. The set of final states F,» includes all states in F4 as well as the 
start state s,+, because A must be recognized. To recognize concatenations of arbitrarily many 
strings from A, we include all the transitions in M,, as well as transitions from s,» that match 
the transitions from s4, and transitions from each final state that match the transitions from s4. 
With this set of transitions, a string made up of concatenations of strings from A will take s,» 
to a final state when the first string in A has been read, returning to a final state when the second 
string in A has been read, and so on. Figure 2(c) illustrates the construction we used. < 


A nondeterministic finite-state automaton can be constructed for any regular set using the 
procedure described in this proof. We illustrate how this is done with Example 3. 


Construct a nondeterministic finite-state automaton that recognizes the regular set 1* U 01. 


Solution: We begin by building a machine that recognizes 1*. This is done using the machine 
that recognizes 1 and then using the construction for M,» described in the proof. Next, we build 
a machine that recognizes 01, using machines that recognize 0 and 1 and the construction in the 
proof for M,,. Finally, using the construction in the proof for M42, we construct the machine 
for 1* U 01. The finite-state automata used in this construction are shown in Figure 3. The states 
in the successive machines have been labeled using different subscripts, even when a state is 
formed from one previously used in another machine. Note that the construction given here 
does not produce the simplest machine that recognizes 1* U01. A much simpler machine that 
recognizes this set is shown in Figure 3(b). < 


13.4.3 Regular Sets and Regular Grammars 


In Section 13.1 we introduced phrase-structure grammars and defined different types of gram- 
mars. In particular we defined regular, or type 3, grammars, which are grammars of the 
form G = (V, T, S, P), where each production is of the form S > A, A > a, or A > aB, where a 
is a terminal symbol, and A and B are nonterminal symbols. As the terminology suggests, there 
is a close connection between regular grammars and regular sets. 


A set is generated by a regular grammar if and only if it is a regular set. 


Proof: First we show that a set generated by a regular grammar is a regular set. Suppose that 
G = (V, T, S, P) is a regular grammar generating the set L(G). To show that L(G) is regular we 
will build a nondeterministic finite-state machine M = (S, I, f, Sọ F) that recognizes L(G). Let 
S, the set of states, contain a state s, for each nonterminal symbol A of G and an additional 
state Sp, which is a final state. The start state sọ is the state formed from the start symbol S. 
The transitions of M are formed from the productions of G in the following way. A transition 


922 13/ Modeling Computation 


(a) Machine Set recognized 


Start q S : 1 1 


0l 


1*U01 


(b) 1*U01 


FIGURE 3 Nondeterministic finite-state automata recognizing 1* u 01. 


from s4 to sp on input of a is included if A > a is a production, and a transition from s4 to sz 
on input of a is included if A > aB is a production. The set of final states includes sp and also 
includes sọ if S > A is a production in G. It is not hard to show that the language recognized 
by M equals the language generated by the grammar G, that is, L(M) = L(G). This can be done 
by determining the words that lead to a final state. The details are left as an exercise for the 
reader. < 


EXAMPLE 4 


FIGURE 4 A 
nondeterministic 
finite-state 
automaton 
recognizing L(G). 


EXAMPLE 5 


FIGURE 5 
A finite-state 
automaton. 


EXAMPLE 6 


? 
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Before giving the proof of the converse, we illustrate how a nondeterministic machine is 
constructed that recognizes the same set as a regular grammar. 


Construct a nondeterministic finite-state automaton that recognizes the language generated by 
the regular grammar G = (V, T, S, P), where V = {0, 1, A, S}, T = {0, 1}, and the productions in 
Pare S > 1A, S > 0, S > å, A > 0A, A > 1A, and A > 1. 


Solution: The state diagram for a nondeterministic finite-state automaton that recognizes L(G) 
is shown in Figure 4. This automaton is constructed following the procedure described in the 
proof. In this automaton, sọ is the state corresponding to S, s, is the state corresponding to A, 
and s, is the final state. < 


We now complete the proof of Theorem 2. 


Proof: We now show that if a set is regular, then there is a regular grammar that generates 
it. Suppose that M is a finite-state machine that recognizes this set with the property that sp, 
the starting state of M, is never the next state for a transition. (We can find such a machine 
by Exercise 20.) The grammar G = (V, T, S, P) is defined as follows. The set V of symbols of 
G is formed by assigning a symbol to each state of S and each input symbol in 7. The set T 
of terminal symbols of G is the set 7. The start symbol S is the symbol formed from the start 
state sy. The set P of productions in G is formed from the transitions in M. In particular, if the 
state s goes to a final state under input a, then the production A, > a is included in P, where 
A, is the nonterminal symbol formed from the state s. If the state s goes to the state ¢ on input 
a, then the production A, > aA, is included in P. The production § — A is included in P if 
and only if A € L(M). Because the productions of G correspond to the transitions of M and the 
productions leading to terminals correspond to transitions to final states, it is not hard to show 
that L(G) = L(M). We leave the details as an exercise for the reader. < 


Example 5 illustrates the construction used to produce a grammar from an automaton that 
generates the language recognized by this automaton. 


Find a regular grammar that generates the regular set recognized by the finite-state automaton 
shown in Figure 5. 


Solution: The grammar G = (V, T, S, P) generates the set recognized by this automaton where 
V = {S, A, B,0, 1}, the symbols S, A, and B correspond to the states sg, s}, and s3, respec- 
tively, T = {0, 1}, S is the start symbol; and the productions are S => 0A, S > 1B, S > 1, 
S > A, A > 0A, A —> 1B, A > 1, B —> OA, B > 1B, and B > 1. < 


13.4.4 A Set Not Recognized by a Finite-State Automaton 


We have seen that a set is recognized by a finite state automaton if and only if it is regular. We 
will now show that there are sets that are not regular by describing one such set. The technique 
used to show that this set is not regular illustrates an important method for showing that certain 
sets are not regular. 


Show that the set {0"1" | n = 0, 1, 2, ... }, made up of all strings consisting of a block of Os 
followed by a block of an equal number of 1s, is not regular. 


Solution: Suppose that this set were regular. Then there would be a nondeterministic finite- 
state automaton M = (S, I, f, Sọ, F) recognizing it. Let N be the number of states in this machine, 
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FIGURE 6 The path produced by 01”. 


that is, N = |S|. Because M recognizes all strings made up of a number of Os followed by an 
equal number of 1s, M must recognize ON14. Let So Sp S2 +++, Soy be the sequence of states 
that is obtained starting at sọ and using the symbols of 0‘1% as input, so that sı = f(so, 0), 
Sy = f(s 0), ..., Sy =f(Sy_1, 9), Syay =S (Sy 1), -5 Soy =f (Soy_y 1). Note that szy is a final 


state. 
Because there are only N states, the pigeonhole principle shows that at least two of the 
first N + 1 of the states, which are sọ, ..., Sy, must be the same. Say that s; and s, are two such 


identical states, with 0 < i < j < N. This means that f(s, 0°) = Sj, where t = j — i. It follows that 
there is a loop leading from s; back to itself, obtained using the input 0 a total of f times, in the 
state diagram shown in Figure 6. 

Now consider the input string 0Y0'1Y = ON¥1^. There are ¢ more consecutive Os at the 
start of this block than there are consecutive 1s that follow it. Because this string is not of the 
form 0" 1” (because it has more Os than 1s), itis not recognized by M. Consequently, f (sọ, ON*1%) 
cannot be a final state. However, when we use the string 0+1” as input, we end up in the same 
state as before, namely, sy. The reason for this is that the extra ¢ Os in this string take us around 
the loop from s; back to itself an extra time, as shown in Figure 6. Then the rest of the string leads 
us to exactly the same state as before. This contradiction shows that {0”1” | n = 0, 1, 2, ... } is 
not regular. < 


ALAN MATHISON TURING (1912-1954) Alan Turing was born in London, although he was conceived in 
India, where his father was employed in the Indian Civil Service. As a boy, he was fascinated by chemistry, 
performing a wide variety of experiments, and by machinery. Turing attended Sherborne, an English board- 
ing school. In 1931 he won a scholarship to King’s College, Cambridge. After completing his dissertation, 
which included a rediscovery of the central limit theorem, a famous theorem in statistics, he was elected a 
fellow of his college. In 1935 Turing became fascinated with the decision problem, a problem posed by the 
great German mathematician Hilbert, which asked whether there is a general method that can be applied to 
any assertion to determine whether the assertion is true. Turing enjoyed running (later in life running as a se- 
rious amateur in competitions), and one day, while resting after a run, he discovered the key ideas needed to 
solve the decision problem. In his solution, he invented what is now called a Turing machine as the most 
general model of a computing machine. Using these machines, he found a problem, involving what he called computable numbers, 
that could not be decided using a general method. 

From 1936 to 1938 Turing visited Princeton University to work with Alonzo Church, who had also solved Hilbert’s decision 
problem. In 1939 Turing returned to King’s College. However, at the outbreak of World War II, he joined the Foreign Office, 
performing cryptanalysis of German ciphers. His contribution to the breaking of the code of the Enigma, a mechanical German 
cipher machine, played an important role in winning the war. 

After the war, Turing worked on the development of early computers. He was interested in the ability of machines to think, 
proposing that if a computer could not be distinguished from a person based on written replies to questions, it should be considered 
to be “thinking.” He was also interested in biology, having written on morphogenesis, the development of form in organisms. In 1954 
Turing committed suicide by taking cyanide, without leaving a clear explanation. Legal troubles related to a homosexual relationship 
and hormonal treatments mandated by the court to lessen his sex drive may have been factors in his decision to end his life. 
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13.4.5 More Powerful Types of Machines 


Finite-state automata are unable to carry out many computations. The main limitation of these 
machines is their finite amount of memory. This prevents them from recognizing languages that 
are not regular, such as {0"1" | n = 0, 1, 2,... }. Because a set is regular if and only if it is the 
language generated by a regular grammar, Example 6 shows that there is no regular grammar 
that generates the set {0”1” | n = 0, 1, 2, ... }. However, there is a context-free grammar that 
generates this set. Such a grammar was given in Example 5 in Section 13.1. 

Because of the limitations of finite-state machines, it is necessary to use other, more power- 
ful, models of computation. One such model is the pushdown automaton. A pushdown automa- 
ton includes everything in a finite-state automaton, as well as a stack, which provides unlimited 
memory. Symbols can be placed on the top or taken off the top of the stack. A set is recognized 
in one of two ways by a pushdown automaton. First, a set is recognized if the set consists of all 
the strings that produce an empty stack when they are used as input. Second, a set is recognized 
if it consists of all the strings that lead to a final state when used as input. It can be shown that 
a set is recognized by a pushdown automaton if and only if it is the language generated by a 
context-free grammar. 

However, there are sets that cannot be expressed as the language generated by a context- 
free grammar. One such set is {0”1"2" | n = 0, 1, 2, ... }. We will indicate why this set cannot 
be recognized by a pushdown automaton, but we will not give a proof, because we have not 
developed the machinery needed. (However, one method of proof is given in Exercise 28 of the 
supplementary exercises at the end of this chapter.) The stack can be used to show that a string 
begins with a sequence of Os followed by an equal number of 1s by placing a symbol on the 
stack for each 0 (as long as only Os are read), and removing one of these symbols for each 1 (as 
long as only 1s following the Os are read). But once this is done, the stack is empty, and there is 
no way to determine that there are the same number of 2s in the string as Os. 

There are other machines called linear bounded automata, more powerful than push- 
down automata, that can recognize sets such as {0"1"2” | n = 0, 1, 2, ... }. In particular, linear 
bounded automata can recognize context-sensitive languages. However, these machines cannot 
recognize all the languages generated by phrase-structure grammars. To avoid the limitations 
of special types of machines, the model known as a Turing machine, named after the British 
mathematician Alan Turing, is used. A Turing machine is made up of everything included in a 
finite-state machine together with a tape, which is infinite in both directions. A Turing machine 


existed! has read and write capabilities on the tape, and it can move back and forth along this tape. Tur- 
ing machines can recognize all languages generated by phrase-structure grammars. In addition, 
Turing machines can model all the computations that can be performed on a computing ma- 
chine. Because of their power, Turing machines are extensively studied in theoretical computer 
science. We will briefly study them in Section 13.5. 
Exercises 
1. Describe in words the strings in each of these regular sets. e) (01)*(11)* f) 0*(10U11)* 
a) 1°0 b) 1°00 g) 0*(10)*11 h) 01(01 U 0)1* 
c) 111 u 001 d) (1 u 00)* 4. Determine whether 1011 belongs to each of these regular 
e) (00*1)* f) (0u 1)(0 u 1)*00 sets. 
2. Describe in words the strings in each of these regular sets. a) 101" E b) ode U 11)" 
a) 001" b) (01)* c) 1(01) 1 ; d) 1 010 U 1) 
c) O1U 001" d) 0(11 U0)* OY eae ee 
i ; g) (10)*1011 h) (1 u 00)(01 u 0)1* 
e) (101*)* f) (0*U1)11 5 h of th : l : 
3. Determine whether 0101 belongs to each of these regular ee ee ee 
g t 8 8 a) the set consisting of the strings 0, 11, and 010 
a b) the set of strings of three Os followed by two or 
a) 01°0* b) 0(11)*(01)* aie 0s 


c) 0(10)*1* 


d) 0*100 U1) c) the set of strings of odd length 
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d) the set of strings that contain exactly one 1 
e) the set of strings ending in 1 and not containing 000 


. Express each of these sets using a regular expression. 


a) the set containing all strings with zero, one, or two 
bits 

b) the set of strings of two Os, followed by zero or 
more 1s, and ending with a 0 

c) the set of strings with every 1 followed by two Os 

d) the set of strings ending in 00 and not containing 11 

e) the set of strings containing an even number of 1s 


. Express each of these sets using a regular expression. 


a) the set of strings of one or more Os followed by a 1 

b) the set of strings of two or more symbols followed by 
three or more Os 

c) the set of strings with either no 1 preceding a 0 or 
no 0 preceding a 1 

d) the set of strings containing a string of 1s such that 
the number of 1s equals 2 modulo 3, followed by an 
even number of Os 


. Construct deterministic finite-state automata that recog- 


nize each of these sets from /*, where / is an alphabet. 
a) Ø b) {A} c) {a}, where a E I 


. Construct nondeterministic finite-state automata that rec- 


ognize each of the sets in Exercise 8. 


. Construct nondeterministic finite-state automata that rec- 


ognize each of these sets. 


a) {4,0} b) {0,11} c) {0, 11, 000} 


Show that if A is a regular set, then AF, the set of all re- 
versals of strings in A, is also regular. 


Using the constructions described in the proof of 
Kleene’s theorem, find nondeterministic finite-state au- 
tomata that recognize each of these sets. 

a) 01* b) (0 U 1)1* c) 00(1* u 10) 


Using the constructions described in the proof of 
Kleene’s theorem, find nondeterministic finite-state au- 
tomata that recognize each of these sets. 

a) 0*1* b) (0 u 11)* c) 01* u 00*1 
Construct a nondeterministic finite-state automaton 
that recognizes the language generated by the regular 
grammar G = (V, T, S, P), where V = {0, 1, S, A, B}, T = 
{0, 1}, S is the start symbol, and the set of productions is 


a) S > 0A, S > 1B,A > 0,B > 0. 

b) S> 1A, S —> 0, S —> å, A > 0B, B > 1B,B > 1. 

c) S> 1B,S—0,A—14,A—=>0B,A>1l,A—> 0, 
Bol. 


In Exercises 15-17 construct a regular grammar G = 
(V, T, S, P) that generates the language recognized by the given 
finite-state machine. 


15. 


0, 1 


Start So 0 0,1 


16. 


18. 
19. 
20. 


*21. 


*22. 


Links > 


* 23. 
*24. 


* 25, 


** 26, 


Show that the finite-state automaton constructed from a 
regular grammar in the proof of Theorem 2 recognizes 
the set generated by this grammar. 


Show that the regular grammar constructed from a finite- 
state automaton in the proof of Theorem 2 generates the 
set recognized by this automaton. 


Show that every nondeterministic finite-state automaton 
is equivalent to another such automaton that has the prop- 
erty that its starting state is never revisited. 


Let M = (S, I, f, Sọ F) be a deterministic finite-state au- 
tomaton. Show that the language recognized by M, L(M), 
is infinite if and only if there is a word x recognized by 
M with Iœ) > |S]. 


One important technique used to prove that certain sets 
are not regular is the pumping lemma. The pumping 
lemma states that if M = (S, I, f; Sọ F) is a deterministic 
finite-state automaton and if x is a string in L(M), the lan- 
guage recognized by M, with /(x) > |S], then there are 
strings u, v, and w in /* such that x = uvw, [(uv) < |S| and 
Iv) > 1, and uv'w € L(M) for i= 0, 1,2, .... Prove the 
pumping lemma. [Hint: Use the same idea as was used 
in Example 5.] 


Show that the set {0?°"1” | n = 0, 1, 2, ...} is not regular 
using the pumping lemma given in Exercise 22. 


Show that the set {i |n=0, 1, 2, ... } is not regular us- 
ing the pumping lemma from Exercise 22. 


Show that the set of palindromes over {0, 1} is not reg- 
ular using the pumping lemma given in Exercise 22. 
[Hint: Consider strings of the form 0" 10%.) 


Show that a set recognized by a finite-state automaton is 
regular. (This is the if part of Kleene’s theorem.) 


Suppose that L is a subset of /*, where J is a nonempty set 
of symbols. If x € /*, we let L/x = {z € I* | xz € L}. We say 
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I* that are distinguishable with respect to L(M), then 
Fso *) # Flo Y). 


that the strings xel" andyel i are distinguishable with *29. Suppose that L is a subset of /* and for some positive inte- 
respect to L if L/x #L/y. A String z for which xz € L but ger n there are n strings in J* such that every two of these 
yz É L, or xz É L, but yz € L is said to distinguish x and y strings are distinguishable with respect to L. Prove that 
with respect to L. When L/x = L/y, we say that x and y are every deterministic finite-state automaton recognizing L 
indistinguishable with respect to L. has at least n states. 
27. Let L be the set of all bit strings that end with 01. Show a Pa Ln n ee ai ano es at "o ‘ bits EN 
that 11 and 10 are distinguishable with respect to L and MEN SY MVO AOM MIE ENG Asal Use eRe Gee [O STOW 
i E eea . that a deterministic finite-state machine recognizing L 
that the strings 1 and 11 are indistinguishable with re- 2 n 
spect to L. must have at least 2” states. 
* 31. Use Exercise 29 to show that the language consisting of 


28. Suppose that M = (S, L f, Sọ F) is a deterministic finite- 
state machine. Show that if x and y are two strings in 


all bit strings that are palindromes (that is, strings that 
equal their own reversals) is not regular. 


Turing Machines 
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Turing 


Definition 1 


13.5.1 Introduction 


The finite-state automata studied earlier in this chapter cannot be used as general models of 
computation. They are limited in what they can do. For example, finite-state automata are able 
to recognize regular sets, but are not able to recognize many easy-to-describe sets, including 
{0"1" | n > 0}, which computers recognize using memory. We can use finite-state automata to 
compute relatively simple functions such as the sum of two numbers, but we cannot use them 
to compute functions that computers can, such as the product of two numbers. To overcome 
these deficiencies we can use a more powerful type of machine known as a Turing machine, 
after Alan Turing, the illustrious mathematician and computer scientist who invented them in 
the 1930s. 

Basically, a Turing machine consists of a control unit, which at any step is in one of finitely 
many different states, together with a tape divided into cells, which is infinite in both directions. 
Turing machines have read and write capabilities on the tape as the control unit moves back and 
forth along this tape, changing states depending on the tape symbol read. Turing machines are 
more powerful than finite-state machines because they include memory capabilities that finite- 
state machines lack. We will show how to use Turing machines to recognize sets, including 
sets that cannot be recognized by finite-state machines. We will also show how to compute 
functions using Turing machines. Turing machines are the most general models of computation; 
essentially, they can do whatever a computer can do. Note that Turing machines are much more 
powerful than real computers, which have finite memory capabilities. 


13.5.2 Definition of Turing Machines 


We now give the formal definition of a Turing machine. Afterward we will explain how this 
formal definition can be interpreted in terms of a control head that can read and write symbols 
on a tape and move either right or left along the tape. 


A Turing machine T = (S, l, f, Sọ) consists of a finite set S of states, an alphabet 7 containing 
the blank symbol B, a partial function f from S x J to Sx I x {R, L}, and a starting state sọ. 


Recall from Section 2.3 that a partial function is defined only for those elements in its do- 
main of definition. This means that for some (state, symbol) pairs the partial function f may be 
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EXAMPLE 1 


Extra 
Examples 
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Control 54 s 
Unit —" ` 
S 
~+— Read/Write Head 
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Tape is infinite in both directions. 
Only finitely many nonblank cells at any time. 


FIGURE 1 A representation of a Turing machine. 


undefined, but for a pair for which it is defined, there is a unique (state, symbol, direction) 
triple associated to this pair. We call the five-tuples corresponding to the partial function in the 
definition of a Turing machine the transition rules of the machine. 

To interpret this definition in terms of a machine, consider a control unit and a tape divided 
into cells, infinite in both directions, having only a finite number of nonblank symbols on it at 
any given time, as pictured in Figure 1. The action of the Turing machine at each step of its 
operation depends on the value of the partial function f for the current state and tape symbol. 

At each step, the control unit reads the current tape symbol x. If the control unit is in state s 
and if the partial function f is defined for the pair (s, x) with f(s, x) = (s’, x’, d), the control unit 


1. enters the state s’, 
2. writes the symbol x’ in the current cell, erasing x, and 
3. moves right one cell if d = R or moves left one cell if d = L. 


We write this step as the five-tuple (s, x, s’, x’, d). If the partial function f is undefined for the 
pair (s, x), then the Turing machine T will halt. 

A common way to define a Turing machine is to specify a set of five-tuples of the form 
(s, x, s’, x’, d). The set of states and input alphabet is implicitly defined when such a definition is 
used. 

At the beginning of its operation a Turing machine is assumed to be in the initial state sọ 
and to be positioned over the leftmost nonblank symbol on the tape. If the tape is all blank, the 
control head can be positioned over any cell. We will call the positioning of the control head 
over the leftmost nonblank tape symbol the initial position of the machine. 

Example 1 illustrates how a Turing machine works. 


What is the final tape when the Turing machine T defined by the seven five-tuples (sp, 0, Sọ 0, R), 
(sg, 1, 51, 1, R), (So, B, 53, B, R), (51, 0, 59, 0, R), (si L, 55, 0, L) (s1, B, 53, B, R), and (s3, 1, 53, 0, R) 
is run on the tape shown in Figure 2(a)? 


Solution: We start the operation with T in state sy and with T positioned over the leftmost 
nonblank symbol on the tape. The first step, using the five-tuple (sp, 0, Sọ, 0, R), reads the O in 
the leftmost nonblank cell, stays in state sọ, writes a O in this cell, and moves one cell right. The 
second step, using the five-tuple (so, 1, s,, 1, R), reads the 1 in the current cell, enters state s,, 
writes a 1 in this cell, and moves one cell right. The third step, using the five-tuple (s4, 0, Sọ, 0, R), 
reads the O in the current cell, enters state sọ, writes a O in this cell, and moves one cell right. 
The fourth step, using the five-tuple (so, 1, s1, 1, R), reads the 1 in the current cell, enters state s,, 
writes a 1 in this cell, and moves right one cell. The fifth step, using the five-tuple (s4, 1, 55, 0, L), 
reads the 1 in the current cell, enters state sj, writes a 0 in this cell, and moves left one cell. The 
sixth step, using the five-tuple (s2, 1, s3, 0, R), reads the 1 in the current cell, enters the state s3, 
writes a O in this cell, and moves right one cell. Finally, in the seventh step, the machine halts 
because there is no five-tuple beginning with the pair (s3, 0) in the description of the machine. 
The steps are shown in Figure 2. 

Note that T changes the first pair of consecutive 1s on the tape to Os and then halts. < 
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(a) So Initial Position 


(b) = 


(c) Sy 


(d) m 


®© s2 


(g) 53 


B B 0 1 0 0 0 0 B B 


Machine Halts 


FIGURE 2 The steps produced by running T on the tape in Figure 1. 


13.5.3 Using Turing Machines to Recognize Sets 


Turing machines can be used to recognize sets. To do so requires that we define the concept of 
a final state as follows. A final state of a Turing machine T is a state that is not the first state in 
any five-tuple in the description of T using five-tuples (for example, state s; in Example 1). 

We can now define what it means for a Turing machine to recognize a string. Given a string, 
we write consecutive symbols in this string in consecutive cells. 


Let V be a subset of an alphabet J. A Turing machine T = (S, I, f, Sọ) recognizes a string x in 
V* if and only if T, starting in the initial position when x is written on the tape, halts in a final 
state. T is said to recognize a subset A of V* if x is recognized by T if and only if x belongs 
to A. 


Note that to recognize a subset A of V* we can use symbols not in V. This means that the input 
alphabet J may include symbols not in V. These extra symbols are often used as markers (see 
Example 3). 
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EXAMPLE 2 


EXAMPLE 3 


? 


When does a Turing machine T not recognize a string x in V*? The answer is that x is 
not recognized if T does not halt or halts in a state that is not final when it operates on a tape 
containing the symbols of x in consecutive cells, starting in the initial position. (The reader 
should understand that this is one of many possible ways to define how to recognize sets using 
Turing machines.) 

We illustrate this concept with Example 2. 


Find a Turing machine that recognizes the set of bit strings that have a 1 as their second bit, that 
is, the regular set (0 u 1)1(0 u 1)*. 


Solution: We want a Turing machine that, starting at the leftmost nonblank tape cell, moves 
right, and determines whether the second symbol is a 1. If the second symbol is 1, the machine 
should move into a final state. If the second symbol is not a 1, the machine should not halt or it 
should halt in a nonfinal state. 

To construct such a machine, we include the five-tuples (sọ, 0, s1, 0, R) and (so, 1, 51, 1, R) to 
read in the first symbol and put the Turing machine in state s,. Next, we include the five-tuples 
(5), 0, 55, 0, R) and (s,, 1, 53, 1, R) to read in the second symbol and either move to state s, if this 
symbol is a 0, or to state s, if this symbol is a 1. We do not want to recognize strings that have a 
0 as their second bit, so s, should not be a final state. We want s, to be a final state. So, we can 
include the five-tuple (s5, 0, s2, 0, R). Because we do not want to recognize the empty string or 
a string with one bit, we also include the five-tuples (so, B, S2, 0, R) and (s,, B, 5, 0, R). 

The Turing machine 7 consisting of the seven five-tuples listed here will terminate in the 
final state s} if and only if the bit string has at least two bits and the second bit of the input string 
is a 1. If the bit string contains fewer than two bits or if the second bit is not a 1, the machine 
will terminate in the nonfinal state sz. < 


Given a regular set, a Turing machine that always moves to the right can be built to recognize 
this set (as in Example 2). To build the Turing machine, first find a finite-state automaton that 
recognizes the set and then construct a Turing machine using the transition function of the 
finite-state machine, always moving to the right. 

We will now show how to build a Turing machine that recognizes a nonregular set. 


Find a Turing machine that recognizes the set {0"1" | n > 1}. 


Solution: To build such a machine, we will use an auxiliary tape symbol M as a marker. We 
have V = {0,1} and Z = {0, 1, M}. We wish to recognize only a subset of strings in V*. We will 
have one final state, sę. The Turing machine successively replaces a 0 at the leftmost position 
of the string with an M and a 1 at the rightmost position of the string with an M, sweeping back 
and forth, terminating in a final state if and only if the string consists of a block of Os followed 
by a block of the same number of 1s. 

Although this is easy to describe and is easily carried out by a Turing machine, the machine 
we need to use is somewhat complicated. We use the marker M to keep track of the leftmost 
and rightmost symbols we have already examined. The five-tuples we use are (sọ, 0, s}, M, R), 
(51, 0, s, 0, R), (si, 1, 51, 1, R), (s1, M, S2, M, L), (S1, B, S2, B, L), (s2, 1, 53, M, L), (53, 1, 53, 1, L), 
(s3, 0, s4, 0, L), (s3, M, S5, M, R), (s4 0, s4, 0, L), (S4, M, So, M, R), and (s5, M, Sẹ M, R). For exam- 
ple, the string 000111 would successively become M00111, M0011M, MM011M, MMOIMM, 
MMM1MM, MMMMMM as the machine operates until it halts. Only the changes are shown, as 
most steps leave the string unaltered. 

We leave it to the reader (Exercise 13) to explain the actions of this Turing machine and to 
explain why it recognizes the set {0"1” | n > 1}. < 


It can be shown that a set can be recognized by a Turing machine if and only if it can be 
generated by a type 0 grammar, or in other words, if the set is generated by a phrase-structure 
grammar. The proof will not be presented here. 


EXAMPLE 4 
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13.5.4 Computing Functions with Turing Machines 


A Turing machine can be thought of as a computer that finds the values of a partial function. 
To see this, suppose that the Turing machine T, when given the string x as input, halts with the 
string y on its tape. We can then define T(x) = y. The domain of T is the set of strings for which 
T halts; T(x) is undefined if T does not halt when given x as input. Thinking of a Turing machine 
as a machine that computes the values of a function on strings is useful, but how can we use 
Turing machines to compute functions defined on integers, on pairs of integers, on triples of 
integers, and so on? 

To consider a Turing machine as a computer of functions from the set of k-tuples of 
nonnegative integers to the set of nonnegative integers (such functions are called number- 
theoretic functions), we need a way to represent k-tuples of integers on a tape. To do so, 
we use unary representations of integers. We represent the nonnegative integer n by a string 
of n+ 1 1s so that, for instance, 0 is represented by the string 1 and 5 is represented by the 
string 111111. To represent the k-tuple (1,, m, ...,n,), we use a string of n; + 1 1s, followed 
by an asterisk, followed by a string of n, + 1 1s, followed by an asterisk, and so on, end- 
ing with a string of n, + 1 1s. For example, to represent the four-tuple (2, 0, 1, 3) we use the 
string]11 * 1 * 11 * 1111. 

We can now consider a Turing machine T as computing a sequence of number-theoretic 
functions T, TŻ, ..., TK, .... The function T* is defined by the action of T on k-tuples of integers 
represented by unary representations of integers separated by asterisks. 


Construct a Turing machine for adding two nonnegative integers. 


Solution: We need to build a Turing machine T that computes the function f(m,,,) =n, + 
ny. The pair (n,, nz) is represented by a string of n, + 1 1s followed by an asterisk followed 
by n, + 1 1s. The machine T should take this as input and produce as output a tape with n, + 
n, +1 1s. One way to do this is as follows. The machine starts at the leftmost 1 of the input 
string, and carries out steps to erase this 1, halting if n} = 0 so that there are no more 1s before 
the asterisk, replaces the asterisk with the leftmost remaining 1, and then halts. We can use 
these five-tuples to do this: (sọ, 1, sı, B, R), (s1; *, 53, B, R), (si 1, 59, B, R), (s2, 1, s2, 1, R), and 
(85, *, 53, 1, R). < 


Unfortunately, constructing Turing machines to compute relatively simple functions can 
be extremely demanding. For example, one Turing machine for multiplying two nonnegative 
integers found in many books has 31 five-tuples and 11 states. If it is challenging to construct 
Turing machines to compute even relatively simple functions, what hope do we have of building 
Turing machines for more complicated functions? One way to simplify this problem is to use a 
multitape Turing machine that uses more than one tape simultaneously and to build up multitape 
Turing machines for the composition of functions. It can be shown that for any multitape Turing 
machine there is a one-tape Turing machine that can do the same thing. 


13.5.5 Different Types of Turing Machines 


There are many variations on the definition of a Turing machine. We can expand the capabilities 
of a Turing machine in a wide variety of ways. For example, we can allow a Turing machine to 
move right, left, or not at all at each step. We can allow a Turing machine to operate on multiple 
tapes, using (2 + 3n)-tuples to describe the Turing machine when n tapes are used. We can allow 
the tape to be two-dimensional, where at each step we move up, down, right, or left, not just right 
or left as we do on a one-dimensional tape. We can allow multiple tape heads that read different 
cells simultaneously. Furthermore, we can allow a Turing machine to be nondeterministic, by 
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allowing a (state, tape symbol) pair to possibly appear as the first elements in more than one five- 
tuple of the Turing machine. So, unlike a deterministic Turing machine, which has at most one 
prescribed action for a given (state, tape symbol) pair, a nondeterministic Turing machine has 
a set of prescribed actions for such a pair. Hence, a nondeterministic Turing machine branches 
into multiple copies as it runs, each of which follows a sequence of possible transitions. We can 
also reduce the capabilities of a Turing machine in different ways. For example, we can restrict 
the tape to be infinite in only one direction or we can restrict the tape alphabet to have only two 
symbols. All these variations of Turing machines have been studied in detail. 

The crucial point is that no matter which of these variations we use, or even which combi- 
nation of variations we use, we never increase or decrease the power of the machine. Anything 
that one of these variations can do can be done by the Turing machine defined in this section, 
and vice versa. The reason that these variations are useful is that sometimes they make doing 
some particular job much easier than if the Turing machine defined in Definition 1 were used. 
They never extend the capability of the machine. Sometimes it is useful to have a wide variety 
of Turing machines with which to work. For example, one way to show that for every nonde- 
terministic Turing machine, there is a deterministic Turing machine that recognizes the same 
language is to use a deterministic Turing machine with three taps. (For details on variations of 
Turing machines and demonstrations of their equivalence, see [HoMoUIO1].) 

Besides introducing the notion of a Turing machine, Turing also showed that it is possible to 
construct a single Turing machine that can simulate the computations of every Turing machine 
when given an encoding of this target Turing machine and its input. Such a machine is called a 
universal Turing machine. (See a book on the theory of computation, such as [Si06], for more 
about universal Turing machines.) 


13.5.6 The Church—Turing Thesis 


Turing machines are relatively simple. They can have only finitely many states and they can read 
and write only one symbol at a time on a one-dimensional tape. But it turns out that Turing ma- 
chines are extremely powerful. We have seen that Turing machines can be built to add numbers 
and to multiply numbers. Although it may be difficult to actually construct a Turing machine 
to compute a particular function that can be computed with an algorithm, such a Turing ma- 
chine can always be found. This was the original goal of Turing when he invented his machines. 
Furthermore, there is a tremendous amount of evidence for the Church-Turing thesis, which 
states that given any problem that can be solved with an effective algorithm, there is a Turing 
machine that can solve this problem. The reason this is called a thesis rather than a theorem is 
that the concept of solvability by an effective algorithm is informal and imprecise, as opposed 
to the notion of solvability by a Turing machine, which is formal and precise. Certainly, though, 
any problem that can be solved using a computer with a program written in any language, per- 
haps using an unlimited amount of memory, should be considered effectively solvable. (Note 
that Turing machines have unlimited memory, unlike computers in the real world, which have 
only a finite amount of memory.) 

Many different formal theories have been developed to capture the notion of effective com- 
putability. These include Turing’s theory and Church’s lambda-calculus, as well as theories 
proposed by Stephen Kleene and by E. L. Post. These theories seem quite different on the sur- 
face. The surprising thing is that they can be shown to be equivalent by demonstrating that they 
define exactly the same class of functions. With this evidence, it seems that Turing’s original 
ideas, formulated before the invention of modern computers, describe the ultimate capabili- 
ties of these machines. The interested reader should consult books on the theory of compu- 
tation, such as [HoMoUIO01] and [S196], for a discussion of these different theories and their 
equivalence. 

For the remainder of this section we will briefly explore some of the consequences of the 
Church—Turing thesis and we will describe the importance of Turing machines in the study of the 
complexity of algorithms. Our goal will be to introduce some important ideas from theoretical 
computer science to entice the interested student to further study. We will cover a lot of ground 
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quickly without providing explicit details. Our discussion will also tie together some of the 
concepts discussed in previous parts of the book with the theory of computation. 


13.5.7 Computational Complexity, Computability, and 
Decidability 


Throughout this book we have discussed the computational complexity of a wide variety of 
problems. We described the complexity of these problems in terms of the number of operations 
used by the most efficient algorithms that solve them. The basic operations used by algorithms 
differ considerably; we have measured the complexity of different algorithms in terms of bit op- 
erations, comparisons of integers, arithmetic operations, and so on. In Section 3.3, we defined 
various Classes of problems in terms of their computational complexity. However, these defini- 
tions were not precise, because the types of operations used to measure their complexity vary 
so drastically. Turing machines provide a way to make the concept of computational complexity 
precise. If the Church—Turing thesis is true, it would then follow that if a problem can be solved 
using an effective algorithm, then there is a Turing machine that can solve this problem. When 
a Turing machine is used to solve a problem, the input to the problem is encoded as a string of 
symbols that is written on the tape of this Turing machine. How we encode input depends on 
the domain of this input. For example, as we have seen, we can encode a positive integer using 
a string of 1s. We can also devise ways to express pairs of integers, negative integers, and so 
on. Similarly, for graph algorithms, we need a way to encode graphs as strings of symbols. This 
can be done in many ways and can be based on adjacency lists or adjacency matrices. (We omit 
the details of how this is done.) However, the way input is encoded does not matter as long as it 
is relatively efficient, as a Turing machine can always change one encoding into another encod- 
ing. We will now use this model to make precise some of the notions concerning computational 
complexity that were informally introduced in Section 3.3. 

The kind of problems that are most easily studied by using Turing machines are those prob- 
lems that can be answered either by a “yes” or by a “no.” 


A decision problem asks whether statements from a particular class of statements are true. 
Decision problems are also known as yes-or-no problems. 


Given a decision problem, we would like to know whether there is an algorithm that can 
determine whether statements from the class of statements it addresses are true. For example, 
consider the class of statements each of which asks whether a particular integer n is prime. 
This is a decision problem because the answer to the question “Is n prime?” is either yes or no. 
Given this decision problem, we can ask whether there is an algorithm that can decide whether 
each of the statements in the decision problem is true, that is, given an integer n, deciding 
whether n is prime. The answer is that there is such an algorithm. In particular, in Section 3.5 
we discussed the algorithm that determines whether a positive integer n is prime by checking 
whether it is divisible by primes not exceeding its square root. (There are many other algorithms 
for determining whether a positive integer is prime.) 

The set of inputs for which the answer to the yes—no problem is “yes” is a subset of the set 
of possible inputs, that is, it is a subset of the set of strings of the input alphabet. In other words, 
solving a yes—no problem is the same as recognizing the language consisting of all bit strings 
that represent input values to the problem leading to the answer “yes.” Consequently, solving a 
yes—no problem is the same as recognizing the language corresponding to the input values for 
which the answer to the problem is “yes.” 


DECIDABILITY When there is an effective algorithm that decides whether instances of a 
decision problem are true, we say that this problem is solvable or decidable. For instance, the 
problem of determining whether a positive integer is prime is a solvable problem. However, if no 
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effective algorithm exists for solving a problem, then we say the problem is unsolvable or unde- 
cidable. To show that a decision problem is solvable we need only construct an algorithm that 
can determine whether statements of the particular class are true. On the other hand, to show 
that a decision problem is unsolvable we need to prove that no such algorithm exists. (The fact 
that we tried to find such an algorithm but failed, does not prove the problem is unsolvable.) 

By studying only decision problems, it may seem that we are studying only a small set of 
problems of interest. However, most problems can be recast as decision problems. Recasting the 
types of problems we have studied in this book as decision problems can be quite complicated, 
so we will not go into the details of this process here. The interested reader can consult references 
on the theory of computation, such as [Wo87], which, for example, explains how to recast the 
traveling salesperson problem (described in Section 9.6) as a decision problem. (To recast the 
traveling salesman problem as a decision problem, we first consider the decision problem that 
asks whether there is a Hamilton circuit of weight not exceeding k, where k is a positive integer. 
With some additional effort it is possible to use answers to this question for different values of 
k to find the smallest possible weight of a Hamilton circuit.) 

In Section 3.1 we introduced the halting problem and proved that it is an unsolvable prob- 
lem. That discussion was somewhat informal because the notion of a procedure was not precisely 
defined. A precise definition of the halting problem can be made in terms of Turing machines. 


The halting problem is the decision problem that asks whether a Turing machine T eventually 
halts when given an input string x. 


With this definition of the halting problem, we have Theorem 1. 


The halting problem is an unsolvable decision problem. That is, no Turing machine exists 
that, when given an encoding of a Turing machine T and its input string x as input, can 
determine whether T eventually halts when started with x written on its tape. 


The proof of Theorem 1 given in Section 3.1 for the informal definition of the halting problem 
still applies here. 
Other examples of unsolvable problems include: 


(i) the problem of determining whether two context-free grammars generate the same set 

of strings; 

(ii) the problem of determining whether a given set of tiles can be used with repetition 
allowed to cover the entire plane without overlap; and 

(iii) Hilbert’s Tenth Problem, which asks whether there are integer solutions to a given poly- 
nomial equation with integer coefficients. (This question occurs tenth on the influential 
list of 23 unsolved problems Hilbert posed in 1900. Hilbert envisioned that the work 
done to solve these problems would help further the progress of mathematics in the 
twentieth century. The unsolvability of Hilbert’s Tenth Problem was established in 1970 
by Yuri Matiyasevich.) 


COMPUTABILITY A function that can be computed by a Turing machine is called 
computable and a function that cannot be computed by a Turing machine is called uncom- 
putable. It is fairly straightforward, using a countability argument, to show that there are 
number-theoretic functions that are not computable (see Exercise 39 in Section 2.5). However, 
it is not so easy to actually produce such a function. The busy beaver function defined in the 
preamble to Exercise 31 is an example of an uncomputable function. One way to show that 
the busy beaver function is not computable is to show that it grows faster than any computable 
function. (See Exercise 32.) 
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Note that every decision problem can be reformulated as the problem of computing a func- 
tion, namely, the function that has the value 1 when the answer to the problem is “yes” and that 
has the value 0 when the answer to the problem is “no.” A decision problem is solvable if and 
only if the corresponding function constructed in this way is computable. 


THE CLASSES PANDNP In Section 3.3 we informally defined the classes of problems called 
P and NP. We are now able to define these concepts precisely using the notions of deterministic 
and nondeterministic Turing machines. 

We first elaborate on the difference between a deterministic Turing machine and a nonde- 
terministic Turing machine. The Turing machines we have studied in this section have all been 
deterministic. In a deterministic Turing machine T = (S, I, f, Sọ), transition rules are defined by 
the partial function f from S x I to S x I x {R, L}. Consequently, when transition rules of the ma- 
chine are represented as five-tuples of the form (s, x, s’, x’, d), where s is the current state, x is the 
current tape symbol, s’ is the next state, x’ is the symbol that replaces x on the tape, and d is the 
direction the machine moves on the tape, no two transition rules begin with the same pair (s, x). 

In a nondeterministic Turing machine, allowed steps are defined using a relation consisting 
of five-tuples rather than using a partial function. The restriction that no two transition rules 
begin with the same pair (s, x) is eliminated; that is, there may be more than one transition 
rule beginning with each (state, tape symbol) pair. Consequently, in a nondeterministic Turing 
machine, there is a choice of transitions for some pairs of the current state and the tape symbol 
being read. At each step of the operation of a nondeterministic Turing machine, the machine 
picks one of the different choices of the transition rules that begin with the current state and 
tape symbol pair. This choice can be considered to be a “guess” of which step to use. Just 
as for deterministic Turing machines, a nondeterministic Turing machine halts when there is 
no transition rule in its definition that begins with the current state and tape symbol. Given 
a nondeterministic Turing machine T, we say that a string x is recognized by T if and only 
if there exists some sequence of transitions of T that ends in a final state when the machine 
starts in the initial position with x written on the tape. The nondeterministic Turing machine T 
recognizes the set A if x is recognized by T if and only if x € A. The nondeterministic Turing 
machine T is said to solve a decision problem if it recognizes the set consisting of all input 
values for which the answer to the decision problem is yes. 


A decision problem is in P, the class of polynomial-time problems, if it can be solved by a 
deterministic Turing machine in polynomial time in terms of the size of its input. That is, a 
decision problem is in P if there is a deterministic Turing machine T that solves the decision 
problem and a polynomial p(n) such that for all integers n, T halts in a final state after no 
more than p(n) transitions whenever the input to T is a string of length n. A decision problem 
is in NP, the class of nondeterministic polynomial-time problems, if it can be solved by a 
nondeterministic Turing machine in polynomial time in terms of the size of its input. That 
is, a decision problem is in NP if there is a nondeterministic Turing machine T that solves 
the problem and a polynomial p(n) such that for all integers n, T halts for every choice of 
transitions after no more than p(n) transitions whenever the input to T is a string of length n. 


ALONZO CHURCH (1903-1995) Alonzo Church was born in Washington, D.C. He studied at Göttingen 
under Hilbert and later in Amsterdam. He was a member of the faculty at Princeton University from 1927 until 
1967, when he moved to UCLA. Church was one of the founding members of the Association for Symbolic 
Logic. He made many substantial contributions to the theory of computability, including his solution to the 
decision problem, his invention of the lambda-calculus, and his statement of what is now known as the Church— 
Turing thesis. Among Church’s students were Stephen Kleene and Alan Turing. He published articles past his 
90th birthday. 
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1. Let T be the Turing machine defined by the five- 


Problems in P are called tractable, whereas problems not in P are called intractable. For 
a problem to be in P, a deterministic Turing machine must exist that can decide in polynomial 
time whether a particular statement of the class addressed by the decision problem is true. For 
example, determining whether an item is in a list of n elements is a tractable problem. (We 
will not provide details on how this fact can be shown; the basic ideas used in the analyses 
of algorithms earlier in the text can be adapted when Turing machines are employed.) For a 
problem to be in NP, it is necessary only that there be a nondeterministic Turing machine that, 
when given a true statement from the set of statements addressed by the problem, can verify 
its truth in polynomial time by making the correct guess at each step from the set of allowable 
steps corresponding to the current state and tape symbol. The problem of determining whether 
a given graph has a Hamilton circuit is an NP problem, because a nondeterministic Turing 
machine can easily verify that a simple circuit in a graph passes through each vertex exactly 
once. It can do this by making a series of correct guesses corresponding to successively adding 
edges to form the circuit. Because every deterministic Turing machine can also be considered 
to be a nondeterministic Turing machine where each (state, tape symbol) pair occurs in exactly 
one transition rule defining the machine, every problem in P is also in NP. In symbols, P G NP. 

One of the most perplexing open questions in theoretical computer science is whether every 
problem in NP is also in P, that is, whether P = NP. As mentioned in Section 3.3, there is an 
important class of problems, the class of NP-complete problems, such that a problem is in this 
class if it is in the class NP and if it can be shown that if it is also in the class P, then every 
problem in the class NP must also be in the class P. That is, a problem is NP-complete if the 
existence of a polynomial-time algorithm for solving it implies the existence of a polynomial- 
time algorithm for every problem in NP. In this book we have discussed several different NP- 
complete problems, such as determining whether a simple graph has a Hamilton circuit and 
determining whether a proposition in n-variables is a tautology. 
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tuples: (59,0, s1, 1, R), (So, L 51,0, R), (So, B, sı, 0, R), 
(s,,0, 55,1, L), (si 1, s1, 0, R), and (s,, B, s2, O, L). For 
each of these initial tapes, determine the final tape 
when T halts, assuming that T begins in initial 
position. 
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. Let T be the Turing machine defined by the five- 


tuples: (59,0, s1, 0, R), (so l, s1 0, L), (So, B, si 1, R), 
(si 0, s23, 1, R), (8,15, LR), (sı, B, s2, 0, R), and 
(s2, B, 53, O, R). For each of these initial tapes, determine 
the final tape when T halts, assuming that T begins in 
initial position. 
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. What does the Turing machine described by the 


five-tuples (sọ, 0, So 0, R), (Sq, L, 51, 0, R), (Sp, B, S2, B, R), 
(si 0, s1, 0, R), (si, l, Sọ 1, R), and (s4, B, s2, B, R) do 
when given 

a) 11 as input? 

b) an arbitrary bit string as input? 


. What does the Turing machine described by the 


five-tuples (so, 0, 5p, 1, R), (So, 1, Sœ L, R), (Sœ B, S1, B, L), 
(sp 1, sy, 1, R), do when given 

a) 101 as input? 

b) an arbitrary bit string as input? 


. What does the Turing machine described by 


the five-tuples (So, 1, s1, 0, R), (Si bsp LR) 
(si, 0, sy, 0, R), (S2, 0, 53, 1, L), (sz 1, sy, 1, R), 
(s3, 1, s3, 1, L), (s3, 0, s4 0, L), (s4 1, s4 1, L), and 
(s4, 0, Sọ 1, R) do when given 

a) 11 as input? 

b) a bit string consisting entirely of 1s as input? 


. Construct a Turing machine with tape symbols 0, 1, 


and B that, when given a bit string as input, adds a 1 to 
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the end of the bit string and does not change any of the 
other symbols on the tape. 


. Construct a Turing machine with tape symbols 0, 1, 


and B that, when given a bit string as input, replaces the 
first O with a 1 and does not change any of the other sym- 
bols on the tape. 


. Construct a Turing machine with tape symbols 0, 1, 


and B that, given a bit string as input, replaces all Os on 
the tape with 1s and does not change any of the 1s on the 
tape. 


. Construct a Turing machine with tape symbols 0, 1, 


and B that, given a bit string as input, replaces all but 
the leftmost 1 on the tape with Os and does not change 
any of the other symbols on the tape. 


Construct a Turing machine with tape symbols 0, 1, 
and B that, given a bit string as input, replaces the first two 
consecutive 1s on the tape with Os and does not change 
any of the other symbols on the tape. 


Construct a Turing machine that recognizes the set of all 
bit strings that end with a 0. 


Construct a Turing machine that recognizes the set of all 
bit strings that contain at least two 1s. 


Construct a Turing machine that recognizes the set of all 
bit strings that contain an even number of 1s. 


Show at each step the contents of the tape of the Turing 
machine in Example 3 starting with each of these strings. 


a) 0011 b) 00011 c) 101100 d) 000111 


Explain why the Turing machine in Example 3 recog- 
nizes a bit string if and only if this string is of the form 
0” 1” for some positive integer n. 


Construct a Turing machine that recognizes the set 
{07"1" | n> 0}. 

Construct a Turing machine that recognizes the set 
{0"1"2" | n > 0}. 

Construct a Turing machine that computes the function 
f(n) = n + 2 for all nonnegative integers n. 


Construct a Turing machine that computes the function 
f(n)=n-3 ifn > 3 and f(n) = 0 for n = 0, 1, 2 for all 
nonnegative integers n. 


Construct a Turing machine that computes the function 
f(n) = n mod 3 for every nonnegative integer n. 


Construct a Turing machine that computes the function 
f(n) =3 ifn > 5 and f(n) = 0ifn = 0, 1, 2, 3, or 4. 


Construct a Turing machine that computes the function 
f(n) = 2n for all nonnegative integers n. 


Construct a Turing machine that computes the function 
f(n) = 3n for all nonnegative integers n. 


Construct a Turing machine that computes the function 
f(n, Ny) = ny + 2 for all pairs of nonnegative integers n, 
and ny. 


Construct a Turing machine that computes the function 
f(n, m) = min(n,,n,) for all nonnegative integers n, 
and ny. 
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26. Construct a Turing machine that computes the function 
Sf (m,n) =n, +n, +1 for all nonnegative integers n, 
and ny. 


Suppose that T) and T, are Turing machines with disjoint 
sets of states S$, and S, and with transition functions f) and 
Jo, respectively. We can define the Turing machine T,T7,, the 
composite of T, and T,, as follows. The set of states of T; T, 
is S; US). Ti T, begins in the start state of T}. It first executes 
the transitions of T} using f, up to, but not including, the step 
at which T, would halt. Then, for all moves for which 7, halts, 
it executes the same transitions of T, except that it moves to 
the start state of T,. From this point on, the moves of T, T, are 

the same as the moves of T). 

27. By finding the composite of the Turing machines you 
constructed in Exercises 18 and 22, construct a Turing 
machine that computes the function f(n) = 2n + 2. 

28. By finding the composite of the Turing machines you 
constructed in Exercises 18 and 23, construct a Turing 
machine that computes the function f(n) = 3(n + 2) = 
3n +6. 

29. Which of the following problems is a decision prob- 
lem? 

a) What is the smallest prime greater than n? 
b) Is a graph G bipartite? 
c) Given aset of strings, is there a finite-state automaton 


that recognizes this set of strings? 
d) Given a checkerboard and a particular type of poly- 


omino (see Section 1.8), can this checkerboard be 
tiled using polyominoes of this type? 


30. Which of the following problems is a decision prob- 
lem? 

a) Is the sequence a}, dp,..., 
increasing order? 

b) Can the vertices of a simple graph G be colored using 
three colors so that no two adjacent vertices are the 
same color? 

c) What is the vertex of highest degree in a graph G? 

d) Given two finite-state machines, do these machines 
recognize the same language? 


a„ Of positive integers in 


Let B(n) be the maximum number of 1s that a Turing machine 


Links) with n states with the alphabet { 1, B} may print on a tape that 


is initially blank. The problem of determining B(n) for partic- 
ular values of n is known as the busy beaver problem. This 
problem was first studied by Tibor Rado in 1962. Currently 
it is known that B(2) = 4, B(3) = 6, and B(4) = 13, but B(n) 
is not known for n > 5. B(n) grows rapidly; it is known that 
B(5) > 4098 and B(6) > 3.5 x 101867. 
* 31. Show that B(2) is at least 4 by finding a Turing machine 
with two states and alphabet {1, B} that halts with four 
consecutive 1s on the tape. 


** 32. Show that the function B(m) cannot be computed by any 


Turing machine. [Hint: Assume that there is a Turing ma- 
chine that computes B(n) in binary. Build a Turing ma- 
chine T that, starting with a blank tape, writes n down in 
binary, computes B(n) in binary, and converts B(n) from 
binary to unary. Show that for sufficiently large n, the 
number of states of T is less than B(n), leading to a con- 
tradiction. ] 
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Key Terms and Results 


TERMS 


alphabet (or vocabulary): a set that contains elements used 
to form strings 


language: a subset of the set of all strings over an alphabet 


phrase-structure grammar (V, T, S, P): a description of a 
language containing an alphabet V, a set of terminal sym- 
bols T, a start symbol S, and a set of productions P 


the production w > w,: w can be replaced by w, whenever it 
occurs in a string in the language 


W =w, (w, is directly derivable from w4): w, can be ob- 
tained from w, using a production to replace a string in w, 
with another string 


wi > w, (w, is derivable from w,): w, can be obtained from 
w; using a sequence of productions to replace strings by 
other strings 


type 0 grammar: any phrase-structure grammar 


type 1 grammar: a phrase-structure grammar in which every 
production is of the form w; > wz, where w; = lAr and 
w, = lwr, where A E€ N, l, r,w € (NUT) * and w ¥ å, or 
w, = S and w, = 4 as long as S is not on the right-hand 
side of another production 


type 2, or context-free, grammar: a phrase-structure gram- 
mar in which every production is of the form A > w}, 
where A is a nonterminal symbol 


type 3, or regular, grammar: a phrase-structure grammar 
where every production is of the form A > aB, A > a, or 
S — A, where A and B are nonterminal symbols, S is the 
start symbol, and a is a terminal symbol 


derivation (or parse) tree: an ordered rooted tree where the 
root represents the starting symbol of a type 2 grammar, 
internal vertices represent nonterminals, leaves represent 
terminals, and the children of a vertex are the symbols on 
the right side of a production, in order from left to right, 
where the symbol represented by the parent is on the left- 
hand side 


Backus-Naur form: a description of a context-free grammar 
in which all productions having the same nonterminal as 
their left-hand side are combined with the different right- 
hand sides of these productions, each separated by a bar, 
with nonterminal symbols enclosed in angular brackets and 
the symbol > replaced by ::= 


finite-state machine (S, J, O, f, g, Sọ) (or a Mealy machine): 
a six-tuple containing a set S of states, an input alphabet J, 
an output alphabet O, a transition function f that assigns a 
next state to every pair of a state and an input, an output 
function g that assigns an output to every pair of a state and 
an input, and a starting state sy 


AB (concatenation of A and B): the set of all strings formed 
by concatenating a string in A and a string in B in that order 


A* (Kleene closure of A): the set of all strings made up by 
concatenating arbitrarily many strings from A 


deterministic finite-state automaton (S, I, f, S F): a five- 
tuple containing a set S of states, an input alphabet J, a 
transition function f that assigns a next state to every pair 
of a state and an input, a starting state sy, and a set of final 
states F 


nondeterministic finite-state automaton (S, I, f, Sọ F): a 
five-tuple containing a set S of states, an input alphabet J, a 
transition function f that assigns a set of possible next states 
to every pair of a state and an input, a starting state sọ, and 
a set of final states F 


language recognized by an automaton: the set of input 
strings that take the start state to a final state of the 
automaton 


regular expression: an expression defined recursively by 
specifying that Ø, A, and x, for all x in the input alpha- 
bet, are regular expressions, and that (AB), (A UB), and 
A* are regular expressions when A and B are regular 
expressions 


regular set: a set defined by a regular expression (see 
page 820) 


Turing machine T = (S, I, f, Sọ): a four-tuple consisting of a 
finite set S of states, an alphabet J containing the blank sym- 
bol B, a partial function f from S x J to S X I x {R, L}, and 
a Starting state sq 


nondeterministic Turing machine: a Turing machine that 
may have more than one transition rule corresponding to 
each (state, tape symbol) pair 


decision problem: a problem that asks whether statements 
from a particular class of statements are true 


solvable problem: a problem with the property that there is 
an effective algorithm that can solve all instances of the 
problem 


unsolvable problem: a problem with the property that no ef- 
fective algorithm exists that can solve all instances of the 
problem 


computable function: a function whose values can be com- 
puted using a Turing machine 


uncomputable function: a function whose values cannot be 
computed using a Turing machine 


P, the class of polynomial-time problems: the class of prob- 
lems that can be solved by a deterministic Turing machine 
in polynomial time in terms of the size of the input 


NP, the class of nondeterministic polynomial-time prob- 
lems: the class of problems that can be solved by a non- 
deterministic Turing machine in polynomial time in terms 
of the size of the input 


NP-complete: a subset of the class of NP problems with the 
property that if any one of them is in the class P, then all 
problems in NP are in the class P 


RESULTS 


For every nondeterministic finite-state automaton there is 
a deterministic finite-state automaton that recognizes the 
same set. 


Review Questions 


1. a) Define a phrase-structure grammar. 
b) What does it mean for a string to be derivable from a 
string w by a phrase-structure grammar G? 
2. a) What is the language generated by a phrase-structure 
grammar G? 


b) What is the language generated by the grammar G 
with vocabulary {S, 0, 1}, set of terminals T = {0, 1}, 
starting symbol S, and productions S => 0005, S > 1? 


c) Give a phrase-structure grammar that generates the set 
{01" |n=0,1,2,... }. 
3. a) Define a type 1 grammar. 


b) Give an example of a grammar that is not a type 1 
grammar. 


c) Define a type 2 grammar. 


d) Give an example of a grammar that is not a type 2 
grammar but is a type 1 grammar. 


e) Define a type 3 grammar. 
f) Give an example of a grammar that is not a type 3 
grammar but is a type 2 grammar. 
4. a) Define a regular grammar. 
b) Define a regular language. 
c) Show that the set {0’"1” | m,n = 0, 1, 2,... } is a reg- 
ular language. 
5. a) What is Backus—Naur form? 
b) Give an example of the Backus—Naur form of the 
grammar for a subset of English of your choice. 
6. a) What is a finite-state machine? 


b) Show how a vending machine that accepts only quar- 
ters and dispenses a soft drink after 75 cents has been 
deposited can be modeled using a finite-state ma- 
chine. 


Supplementary Exercises 


*1, Find a phrase-structure grammar that generates each of 
these languages. 


a) the set of bit strings of the form 02"13", where n is a 
nonnegative integer 

b) the set of bit strings with twice as many Os as 1s 

c) the set of bit strings of the form w?, where w is a bit 
string 


*2. Find a phrase-structure grammar that generates the set 
{07" |n>0}. 
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Kleene’s theorem: A set is regular if and only if there is a 
finite-state automaton that recognizes it. 


A set is regular if and only if it is generated by a regular 
grammar. 


The halting problem is unsolvable. 


7. Find the set of strings recognized by the deterministic 
finite-state automaton shown here. 


8. Construct a deterministic finite-state automaton that rec- 
ognizes the set of bit strings that start with 1 and end 
with 1. 

9. a) What is the Kleene closure of a set of strings? 

b) Find the Kleene closure of the set {11, 0}. 

10. a) Define a finite-state automaton. 

b) What does it mean for a string to be recognized by a 
finite-state automaton? 

11. a) Define a nondeterministic finite-state automaton. 

b) Show that given a nondeterministic finite-state au- 
tomaton, there is a deterministic finite-state automa- 
ton that recognizes the same language. 

12. a) Define the set of regular expressions over a set J. 

b) Explain how regular expressions are used to represent 
regular sets. 

13. State Kleene’s theorem. 

14. Show that a set is generated by a regular grammar if and 
only if it is a regular set. 

15. Give an example of a set not recognized by a finite-state 
automaton. Show that no finite-state automaton recog- 
nizes it. 

16. Define a Turing machine. 

17. Describe how Turing machines are used to recognize sets. 

18. Describe how Turing machines are used to compute 
number-theoretic functions. 

19. What is an unsolvable decision problem? Give an exam- 
ple of such a problem. 


For Exercises 3 and 4, let G = (V, T, S, P) be the context-free 
grammar with V = {(,),S,A, B}, T = {(,)}, starting sym- 
bol S, and productions S > A,A > AB,A > B, B > (A), and 
BoOQ,S-> A. 


3. Construct the derivation trees of these strings. 
a) (©) b00@) © (OO) 
*4. Show that L(G) is the set of all balanced strings of paren- 


theses, defined in the preamble to Supplementary Exer- 
cise 59 in Chapter 5. 
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A context-free grammar is ambiguous if there is a word in 
L(G) with two derivations that produce different derivation 
trees, considered as ordered, rooted trees. 
5. Show that the grammar G=(V,7,S,P) with V = 
{0,5}, T = {0}, starting state S, and productions $ > 0S, 
S > SO, and S > 0 is ambiguous by constructing two dif- 
ferent derivation trees for 0°. 
6. Show that the grammar G = (V, T, S, P) with V = {0,5}, 
T = {0}, starting state S, and productions S > OS and 
S — 0 is unambiguous. 
7. Suppose that A and B are finite subsets of V*, where V is 
an alphabet. Is it necessarily true that |AB| = |BA|? 
8. Prove or disprove each of these statements for subsets A, 
B, and C of V*, where V is an alphabet. 
a) A(BUC) =ABUAC 
b) ABN C) =ABNAC 
c) (AB)C = A(BC) 
d) (A U B)* = A* U B* 
9. Suppose that A and B are subsets of V*, where V is an 
alphabet. Does it follow that A C B if A* C B*? 
10. What set of strings with symbols in the set {0, 1,2} is 
represented by the regular expression (2*)(0 u (12*))*? 
The star height A(E) of a regular expression over the set J is 
defined recursively by 


h(®) = 0; 

h(x) = Oifx ET; 

h(E, VE,)) = h(E, E,)) = max(A(E,), h(E,)) 
if E, and E, are regular expressions; 

h(E*) = h(E) + 1 if E is a regular expression. 

11. Find the star height of each of these regular expressions. 
a) 0*1 
b) 0*1* 

c) (0*01)* 

d) ((0*1)*)* 

e) (010°)(1*01")*((01)*(10)*)* 
£) ((((O*)1)*0)*)1)" 

* 12. For each of these regular expressions find a regular ex- 
pression that represents the same language with mini- 
mum star height. 

a) (0*1")* 
b) (0(01*0)*)* 
c) (0* u (01)* u 1*)* 

13. Construct a finite-state machine with output that produces 
an output of 1 if the bit string read so far as input contains 
four or more 1s. Then construct a deterministic finite- 
state automaton that recognizes this set. 


14. Construct a finite-state machine with output that produces 
an output of 1 if the bit string read so far as input contains 
four or more consecutive 1s. Then construct a determin- 
istic finite-state automaton that recognizes this set. 


15. Construct a finite-state machine with output that produces 
an output of 1 if the bit string read so far as input ends with 
four or more consecutive 1s. Then construct a determin- 
istic finite-state automaton that recognizes this set. 


16. A state s’ in a finite-state machine is said to be reach- 
able from state s if there is an input string x such that 
f(s,x)= s. A state s is called transient if there is no 
nonempty input string x with f (s, x) = s. A state s is called 
a sink if f(s, x) = s for all input strings x. Answer these 
questions about the finite-state machine with the state di- 
agram illustrated here. 


a) Which states are reachable from sy? 
b) Which states are reachable from s,? 
c) Which states are transient? 

d) Which states are sinks? 


*17. Suppose that S, 7, and O are finite sets such that |S| = n, 


|Z| = k, and |O| = m. 

a) How many different finite-state machines (Mealy 
machines) M = (S, I, O, f, 8, Sy) can be constructed, 
where the starting state sọ can be arbitrarily chosen? 

b) How many different Moore machines M = 
(S, IL, O, f, 8, Sọ) can be constructed, where the start- 
ing state sọ can be arbitrarily chosen? 


*18. Suppose that S and / are finite sets such that |S| = n and 


|Z| =k. How many different finite-state automata M = 
(S, L f; So, F) are there where the starting state sọ and the 
subset F of S consisting of final states can be chosen ar- 
bitrarily 
a) if the automata are deterministic? 
b) if the automata may be nondeterministic? (Note: This 
includes deterministic automata.) 

19. Construct a deterministic finite-state automaton that is 
equivalent to the nondeterministic automaton with the 
state diagram shown here. 


Start 


20. What is the language recognized by the automaton in 
Exercise 19? 

21. Construct finite-state automata that recognize these sets. 
a) 0*(10)* 
b) (01 u 111)*10*(0U 1) 
c) (001 u (11)*)* 


* 22. 


* 23. 
* 24. 
*Z25, 


Find regular expressions that represent the set of all 

strings of Os and 1s 

a) made up of blocks of even numbers of 1s interspersed 
with odd numbers of Os. 

b) with at least two consecutive Os or three consecutive 
Is. 

c) with no three consecutive Os or two consecutive 1s. 


Show that if A is a regular set, then so is A. 
Show that if A and B are regular sets, then so is AN B. 


Find finite-state automata that recognize these sets of 

strings of Os and 1s. 

a) the set of all strings that start with no more than three 
consecutive Os and contain at least two consecutive 1s 

b) the set of all strings with an even number of symbols 
that do not contain the pattern 101 

c) the set of all strings with at least three blocks of two 
or more 1s and at least two Os 


Computer Projects 


Write programs with these input and output. 


1. 


#4, 


Given the productions in a phrase-structure grammar, de- 
termine which type of grammar this is in the Chomsky 
classification scheme. 


. Given the productions of a phrase-structure grammar, 


find all strings that are generated using twenty or fewer 
applications of its production rules. 


. Given the Backus—Naur form of a type 2 grammar, find 


all strings that are generated using twenty or fewer appli- 
cations of the rules defining it. 


Given the productions of a context-free grammar and a 
string, produce a derivation tree for this string if it is in 
the language generated by this grammar. 


. Given the state table of a Moore machine and an input 


string, produce the output string generated by the ma- 
chine. 


. Given the state table of a Mealy machine and an input 


string, produce the output string generated by the ma- 
chine. 


Computations and Explorations 


* 26. 


* 27, 


* 28. 


* 29. 


* 30. 


** 10, 


11. 


12. 


* 13. 
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Show that {0°” | n € N} is not regular. You may use the 
pumping lemma given in Exercise 22 of Section 13.4. 


Show that {1? | p is prime} is not regular. You may use 
the pumping lemma given in Exercise 22 of Section 13.4. 


There is a result for context-free languages analogous to 
the pumping lemma for regular sets. Suppose that L(G) 
is the language recognized by a context-free language G. 
This result states that there is a constant N such that if 
z is a word in L(G) with /(z) > N, then z can be written 
as uvwxy, where I(vwx) < N, I(vx) > 1, and uv'wx'y be- 
longs to L(G) fori =0, 1, 2, 3, .... Use this result to show 
that there is no context-free grammar G with L(G) = 
{0"1"2" |n=0,1,2,... }. 

Construct a Turing machine that computes the function 
f(n, ny) = max(n,, m). 

Construct a Turing machine that computes the function 
Sf (14, Ng) = ny -n ifn, > n andf(n,, m) = Oifny <n. 


. Given the state table of a deterministic finite-state au- 


tomaton and a string, decide whether this string is rec- 
ognized by the automaton. 


. Given the state table of a nondeterministic finite-state au- 


tomaton and a string, decide whether this string is recog- 
nized by the automaton. 


. Given the state table of a nondeterministic finite-state au- 


tomaton, construct the state table of a deterministic finite- 
state automaton that recognizes the same language. 
Given a regular expression, construct a nondeterminis- 
tic finite-state automaton that recognizes the set that this 
expression represents. 

Given a regular grammar, construct a finite-state au- 
tomaton that recognizes the language generated by this 
grammar. 

Given a finite-state automaton, construct a regular gram- 
mar that generates the language recognized by this au- 
tomaton. 

Given a Turing machine, find the output string produced 
by a given input string. 


Use a computational program or programs you have written to do these exercises. 


1. 


*2. 


Solve the busy beaver problem for two states by test- 
ing all possible Turing machines with two states and 
alphabet {1, B}. 


Solve the busy beaver problem for three states by testing 
all possible Turing machines with three states and alphabet 
{1, B}. 


v8 3, 


**4 


**5 


Find a busy beaver machine with four states by testing 
all possible Turing machines with four states and alphabet 
{1, B}. 


. Make as much progress as you can toward finding a busy 


beaver machine with five states. 


. Make as much progress as you can toward finding a busy 


beaver machine with six states. 
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Writing Projects 


Respond to these with essays using outside sources. 


. Describe how the growth of certain types of plants can be 


modeled using a Lidenmeyer system. Such a system uses 
a grammar with productions modeling the different ways 
plants can grow. 


. Describe the Backus—Naur form (and extended Backus— 


Naur form) rules used to specify the syntax of a pro- 
gramming language, such as Java, LISP, or Ada, or the 
database language SQL. 


. Explain how finite-state machines are used by spell- 


checkers. 


. Explain how finite-state machines are used in the study 


of network protocols. 


. Explain how finite-state machines are used in speech 


recognition programs. 


. Compare the use of Moore machines versus Mealy ma- 


chines in the design of hardware systems and computer 
software. 


. Explain the concept of minimizing finite-state automata. 


Give an algorithm that carries out this minimization. 


. Give the definition of cellular automata. Explain their ap- 


plications. Use the Game of Life as an example. 


. Define a pushdown automaton. Explain how pushdown 


automata are used to recognize sets. Which sets are rec- 
ognized by pushdown automata? Provide an outline of a 
proof justifying your answer. 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


Define a linear-bounded automaton. Explain how linear- 
bounded automata are used to recognize sets. Which sets 
are recognized by linear-bounded automata? Provide an 
outline of a proof justifying your answer. 


Look up Turing’s original definition of what we now call 
a Turing machine. What was his motivation for defining 
these machines? 


Describe the concept of the universal Turing machine. 
Explain how such a machine can be built. 


Explain the kinds of applications in which nondetermin- 
istic Turing machines are used instead of deterministic 
Turing machines. 


Show that a Turing machine can simulate any action of a 
nondeterministic Turing machine. 


Show that a set is recognized by a Turing machine if and 
only if it is generated by a phrase-structure grammar. 


Describe the basic concepts of the lambda-calculus and 
explain how it is used to study computability of functions. 


Show that a Turing machine as defined in this chapter can 
do anything a Turing machine with n tapes can do. 


Show that a Turing machine with a tape infinite in one 
direction can do anything a Turing machine with a tape 
infinite in both directions can do. 


and the Positive Integers 


L this book we have assumed an explicit set of axioms for the set of real numbers and for 
the set of positive integers. In this appendix we will list these axioms and we will illustrate 
how basic facts, also used without proof in the text, can be derived using them. 


A1.1 Axioms for Real Numbers 


The standard axioms for real numbers include both the field (or algebraic) axioms, used to 
specify rules for basic arithmetic operations, and the order axioms, used to specify properties 
of the ordering of real numbers. 


THE FIELD AXIOMS We begin with the field axioms. As usual, we denote the sum and 
product of two real numbers x and y by x + y and x - y, respectively. (Note that the product of x 
and y is often denoted by xy without the use of the dot to indicate multiplication. We will not 
use this abridged notation in this appendix, but will within the text.) Also, by convention, we 
perform multiplications before additions unless parentheses are used. Although these statements 
are axioms, they are commonly called laws or rules. The first two of these axioms tell us that 
when we add or multiply two real numbers, the result is again a real number; these are the 
closure laws. 


> Closure law for addition For all real numbers x and y, x + y is a real number. 
> Closure law for multiplication For all real numbers x and y, x - y is a real number. 


The next two axioms tell us that when we add or multiply three real numbers, we get the 
same result regardless of the order of operations; these are the associative laws. 


> Associative law for addition For all real numbers x, y, and z, (xn+y)+z= 
x+(y +2). 
> Associative law for multiplication For all real numbers x, y, and z, (x.y): z= 
x-z). 
Two additional algebraic axioms tell us that the order in which we add or multiply two 
numbers does not matter; these are the commutative laws. 


> Commutative law for addition For all real numbers x and y, x + y = y +x. 
> Commutative law for multiplication For all real numbers x and y, x: y = y- x. 


The next two axioms tell us that 0 and 1 are additive and multiplicative identities for the set 
of real numbers. That is, when we add 0 to a real number or multiply a real number by 1 we do 
not change this real number. These laws are called identity laws. 


> Additive identity law For every real number x, x + 0 = 0 +x = x. 
> Multiplicative identity law For every real number x, x: 1 = 1 -x = x. 


Although it seems obvious, we also need the following axiom. 


> Identity elements axiom The additive identity 0 and the multiplicative identity 1 are 
distinct, that is 0 Æ 1. 
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Two additional axioms tell us that for every real number, there is a real number that can be 
added to this number to produce 0, and for every nonzero real number, there is a real number 
by which it can be multiplied to produce 1. These are the inverse laws. 


> Inverse law for addition For every real number x, there exists a real number —x (called 
the additive inverse of x) such that x + (—x) = (-x) +x = 0. 

> Inverse law for multiplication For every nonzero real number x, there exists a real 
number 1 /x (called the multiplicative inverse of x) such that x -(1/x) = (1/x)-x = 1. 


The final algebraic axioms for real numbers are the distributive laws, which tell us that 
multiplication distributes over addition; that is, that we obtain the same result when we first add 
a pair of real numbers and then multiply by a third real number or when we multiply each of 
these two real numbers by the third real number and then add the two products. 


> Distributive laws For all real numbers x, y, and z, x-(y+z)=x-y+x-z and 
(x+y): Z=x-Z+y-z. 


ORDER AXIOMS Next, we will state the order axioms for the real numbers, which specify 
properties of the “greater than” relation, denoted by >, on the set of real numbers. We write 
x > y (and y < x) when x is greater than y, and we write x > y (and y < x) when x > yor x = y. 
The first of these axioms tells us that given two real numbers, exactly one of three possibilities 
occurs: the two numbers are equal, the first is greater than the second, or the second is greater 
than the first. This rule is called the trichotomy law. 


> Trichotomy law For all real numbers x and y, exactly one of x = y, x > y, or y > x is 
true. 


Next, we have an axiom, called the transitivity law, which tells us that if one number is 
greater than a second number and this second number is greater than a third, then the first 
number is greater than the third. 


> Transitivity law For all real numbers x, y, and z, if x > y and y > z, then x > z. 


We also have two compatibility laws, which tell us that when we add a number to both sides 
in a greater than relationship, the greater than relationship is preserved, and when we multiply 
both sides of a greater than relationship by a positive real number (that is, a real number x with 
x > 0), the greater than relationship is preserved. 


> Additive compatibility law For all real numbers x, y, and z, if x > y, then x + z > y +z. 


> Multiplicative compatibility law For all real numbers x, y, and z, if x > y and z > 0, 
thenx-z>y-z. 


We leave it to the reader (see Exercise 15) to prove that for all real numbers x, y, and z, 
if x > y and z < 0, then x- z < y- z. That is, multiplication of an inequality by a negative real 
number reverses the direction of the inequality. 

The final axiom for the set of real numbers is the completeness property. Before we state 
this axiom, we need some definitions. First, given a nonempty set A of real numbers, we say 
that the real number b is an upper bound of A if for every real number a in A, b > a. A real 
number s is a least upper bound of A if s is an upper bound of A and whenever t is an upper 
bound of A, then we have s < t. 


> Completeness property Every nonempty set of real numbers that is bounded above 
has a least upper bound. 
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A1.2 Using Axioms to Prove Basic Facts 


The axioms we have listed can be used to prove many properties that are often used without 
explicit mention. We give several examples of results we can prove using axioms and leave the 
proof of a variety of other properties as exercises. Although the results we will prove seem quite 
obvious, proving them using only the axioms we have stated can be challenging. 


THEOREM 1 The additive identity element 0 of the real numbers is unique. 


Proof: To show that the additive identity element 0 of the real numbers is unique, suppose that 0’ 
is also an additive identity for the real numbers. This means that 0’ + x = x + 0’ = x whenever 
x is a real number. By the additive identity law, it follows that 0 + 0’ = 0’. Because 0’ is an 
additive identity, we know that 0 + 0’ = 0. It follows that 0 = 0’, because both equal 0 + 0’. 
This shows that 0 is the unique additive identity for the real numbers. < 


THEOREM 2 The additive inverse of a real number x is unique. 


Proof: Let x be a real number. Suppose that y and z are both additive inverses of x. Then, 


y=0+y by the additive identity law 
=(z+x)+y because zis an additive inverse of x 


=Z+(x+y) bythe associative law for addition 


=z+0 because y is an additive inverse of x 
=z by the additive identity law. 
It follows that y = z. < 


Theorems 1 and 2 tell us that the additive identity and additive inverses are unique. Theo- 
rems 3 and 4 tell us that the multiplicative identity and multiplicative inverses of nonzero real 
numbers are also unique. We leave their proofs as exercises. 


THEOREM 3 The multiplicative identity element 1 of the real numbers is unique. 


THEOREM 4 The multiplicative inverse of a nonzero real number x is unique. 


THEOREM 5 For every real number x, x-0 = 0. 


Proof: Suppose that x is a real number. By the additive inverse law, there is a real number y that 
is the additive inverse of x - 0, so we have x -0 + y = 0. By the additive identity law, 0 + 0 = 0. 
Using the distributive law, we see that x-0 = x - (0 +0) = x - 0 + x - 0. It follows that 


O=x-O0+y=(-04+x-0)4+y. 
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THEOREM 6 


THEOREM 7 
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Next, note that by the associative law for addition and because x - 0 + y = 0, it follows that 
(x-0+x-0)+y=x-04+(x-0+y)=x-0+4+0. 


Finally, by the additive identity law, we know that x-0+0=.x-0. Consequently, 
0S 0, < 


For all real numbers x and y, if x - y = 0, then x = 0 or y = 0. 


Proof: Suppose that x and y are real numbers and x - y = 0. If x Æ 0, then, by the multiplica- 
tive inverse law, x has a multiplicative inverse 1/x, such that x - (1/x) = (1/x) - x = 1. Because 
x+y = 0, we have (1/x)-(- y) = (1/x) - 0 = 0 by Theorem 5. Using the associate law for mul- 
tiplication, we have ((1/x) - x) - y = 0. This means that 1 - y = 0. By the multiplicative identity 
rule, we see that 1 - y = y, so y = 0. Consequently, either x = 0 or y = 0. < 


The multiplicative identity element 1 in the set of real numbers is greater than the additive 
identity element 0. 


Proof: By the trichotomy law, either 0 = 1, 0 > 1, or 1 > 0. We know by the identity elements 
axiom that 0 # 1. 

So, assume that 0 > 1. We will show that this assumption leads to a contradiction. By the 
additive inverse law, 1 has an additive inverse —1 with 1 + (—1) = 0. The additive compatibility 
law tells us that 0 + (—1) > 1 + (-1) = 0; the additive identity law tells us that 0 + (—1) = -1. 
Consequently, —1 > 0, and by the multiplicative compatibility law, (—1) - (—1) > (-1) - 0. By 
Theorem 5, the right-hand side of last inequality is 0. By the distributive law, (—1) -(—1) + 
(-1)-1=(-1)-(-1+1) =(-1)-0=0. Hence, the left-hand side of this last inequality, (—1) - 
(—1), is the unique additive inverse of —1, so this side of the inequality equals 1. Consequently, 
this last inequality becomes 1 > 0, contradicting the trichotomy law because we had assumed 
that 0 > 1. 

Because we know that 0 ¥ 1 and that it is impossible for 0 > 1, by the trichotomy law, we 
conclude that 1 > 0. 


ARCHIMEDES (287 B.C.E.—212 B.C.E.) Archimedes was one of the greatest scientists and mathematicians 
of ancient times. He was born in Syracuse, a Greek city-state in Sicily. His father, Phidias, was an astronomer. 
Archimedes was educated in Alexandria, Egypt. After completing his studies, he returned to Syracuse, where 
he spent the rest of his life. Little is known about his personal life; we do not know whether he was ever married 
or had children. Archimedes was killed in 212 B.C.E. by a Roman soldier when the Romans overran Syracuse. 

Archimedes made many important discoveries in geometry. His method for computing the area under 


f=] a curve was described two thousand years before his ideas were re-invented as part of integral calculus. 
S4 Archimedes also developed a method for expressing large integers inexpressible by the usual Greek method. 


He discovered a method for computing the volume of a sphere, as well as of other solids, and he calcu- 
lated an approximation of z. Archimedes was also an accomplished engineer and inventor; his machine for 
pumping water, now called Archimedes’ screw, is still in use today. Perhaps his best known discovery is 
the principle of buoyancy, which tells us that an object submerged in liquid becomes lighter by an amount 
equal to the weight it displaces. Some histories tell us that Archimedes was an early streaker, running naked 


through the streets of Syracuse shouting “Eureka” (which means “I have found it”) when he made this discovery. He is also known 
for his clever use of machines that held off Roman forces sieging Syracuse for several years during the Second Punic War. 
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The next theorem tells us that for every real number there is an integer (where by an integer, 
we mean 0, the sum of any number of 1s, and the additive inverses of these sums) greater than 
this real number. This result is attributed to the Greek mathematician Archimedes. The result 
can be found in Book V of Euclid’s Elements. 


ARCHIMEDEAN PROPERTY For every real number x there exists an integer n such 
that n > x. 


Proof: Suppose that x is a real number such that n < x for every integer n. Then x is an upper 
bound of the set of integers. By the completeness property it follows that the set of integers has 
a least upper bound M. Because M — 1 < M and M is a least upper bound of the set of integers, 
M — 1 is not an upper bound of the set of integers. This means that there is an integer n with 
n > M — 1. This implies that n + 1 > M, contradicting the fact that M is an upper bound of the 
set of integers. < 


A1.3 Axioms for the Set of Positive Integers 


The axioms we now list specify the set of positive integers as the subset of the set of integers 
satisfying four key properties. We assume the truth of these axioms in this textbook. 


> Axiom 1 The number | is a positive integer. 

> Axiom 2 If 7 is a positive integer, then n+ 1, the successor of n, is also a positive 
integer. 

> Axiom 3 Every positive integer other than 1 is the successor of a positive integer. 

> Axiom 4 The Well-Ordering Property Every nonempty subset of the set of positive 
integers has a least element. 


In Sections 5.1 and 5.2 it is shown that the well-ordering principle is equivalent to the 
principle of mathematical induction. 


> Mathematical induction axiom If S is a set of positive integers such that 1 € S$ and 
for all positive integers n if n E€ S, then n + 1 € S, then S is the set of positive integers. 


Most mathematicians take the real number system as already existing, with the real numbers 
satisfying the axioms we have listed in this appendix. However, mathematicians in the nine- 
teenth century developed techniques to construct the set of real numbers, starting with more 
basic sets of numbers. (The process of constructing the real numbers is sometimes studied in 
advanced undergraduate mathematics classes. A treatment of this can be found in [Mo91], for 
instance.) The first step in the process is the construction of the set of positive integers using 
axioms 1-3 and either the well-ordering property or the mathematical induction axiom. Then, 
the operations of addition and multiplication of positive integers are defined. Once this has 
been done, the set of integers can be constructed using equivalence classes of pairs of positive 
integers, where (a, b) ~ (c, d) if and only if a+ d = b+ c; addition and multiplication of inte- 
gers can be defined using these pairs (see Exercise 21). (Equivalence relations and equivalence 
classes are discussed in Chapter 9.) Next, the set of rational numbers can be constructed using 
the equivalence classes of pairs of integers where the second integer in the pair is not zero, 
where (a, b) = (c, d) if and only if a- d = b - c; addition and multiplication of rational numbers 
can be defined in terms of these pairs (see Exercise 22). Using infinite sequences, the set of real 
numbers can then be constructed from the set of rational numbers. The interested reader will 
find it worthwhile to read through the many details of the steps of this construction. 
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Exercises 


Use only the axioms and theorems in this appendix in the 
proofs in your answers to these exercises. 


1. 


2. 


7. 


Prove Theorem 3, which states that the multiplicative 
identity element of the real numbers is unique. 

Prove Theorem 4, which states that for every nonzero real 
number x, the multiplicative inverse of x is unique. 


. Prove that for all real numbers x and y, (—x)-y=x- 


(=y) = —(x-y). 


. Prove that for all real numbers x and y, —(x + y) = 


(=x) + (=y). 


. Prove that for all real numbers x and y, (—x) - (—y) = x - y. 


6. 


Prove that for all real numbers x, y, and z, if x + z = y +z, 
then x = y. 
Prove that for every real number x, —(—x) = x. 


Define the difference x —y of real numbers x and y by 
x— y = x + (—y), where —y is the additive inverse of y, and 
the quotient x/y, where y # 0, by x/y = x- (1/y), where 
1/y is the multiplicative inverse of y. 


8. 


9. 
10. 


11. 


12. 


13. 


14. 


Prove that for all real numbers x and y, x = y if and only 
ifx-y=0. 

Prove that for all real numbers x and y, —x — y = —(x + y). 
Prove that for all nonzero real numbers x and y, 1 /(x/y) = 
y/x, where 1/(x/y) is the multiplicative inverse of x/y. 
Prove that for all real numbers w, x, y, and z, if x # 0 and 
z #0, then (w/x) + (y/z) =(w-z+x-y)/@-2). 

Prove that for every positive real number x, 1 /x is also a 
positive real number. 

Prove that for all positive real numbers x and y, x - y is 
also a positive real number. 

Prove that for all real numbers x and y, if x > O and y < 0, 
thenx-y <0. 


15. 


16. 


17. 


18. 


19. 


* 20. 


Prove that for all real numbers x, y, and z, if x > y and 
z<0,thenx-z<y-z. 


Prove that for every real number x, x 4 0 if and only if 
xX >O. 

Prove that for all real numbers w, x, y, and z, if w < x and 
y<zthenw+y<x4+z. 


Prove that for all positive real numbers x and y, if x < y, 
then 1/x > 1/y. 


Prove that for every positive real number x, there exists a 
positive integer n such that n -x > 1. 


Prove that between every two distinct real numbers there 
is a rational number (that is, a number of the form x/y, 
where x and y are integers with y £ 0). 


Exercises 21 and 22 involve the notion of an equivalence re- 
lation, discussed in Chapter 9 of the text. 


*21. 


*22, 


Define a relation ~ on the set of ordered pairs of posi- 
tive integers by (w, x) ~ Q, z) if and only if w +z = x + 
y. Show that the operations [(w, x)]_ +[0, 2]. = [(w + 
y,x+z)]. and [(w, x]. - 10,2]. =[v-ytx-zx-yt 
w-z)]_ are well-defined, that is, they do not depend on 
the representative of the equivalence classes chosen for 
the computation. 


Define a relation ~ on ordered pairs of integers with 
second entry nonzero by (w,x) *(,z) if and only 
if w-z=x-y. Show that the operations [(w,x)], + 
Ly, Dls = [wz tx-y,x-Z)]~ and [(w, x)]~ 10, De = 
[(w-y, x - z)]„ are well-defined, that is, they do not depend 
on the representative of the equivalence classes chosen 
for the computation. 
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Xponential and Logarithmic 


Functions 


L this appendix we review some of the basic properties of exponential functions and log- 
arithms. These properties are used throughout the text. Students requiring further review 
of this material should consult precalculus or calculus books, such as those mentioned in the 
Suggested Readings. 


A2.1 Exponential Functions 


Let n be a positive integer, and let b be a fixed positive real number. The function f,(n) = b” 
is defined by 


f,(n) =b" =b-b-b- a eb, 


where there are n factors of b multiplied together on the right-hand side of the equation. 

We can define the function f, (x) = b* for all real numbers x using techniques from calculus. 
The function f,(x) = b* is called the exponential function to the base b. We will not discuss 
how to find the values of exponential functions to the base b when x is not an integer. 

Two of the important properties satisfied by exponential functions are given in Theorem 1. 
Proofs of these and other related properties can be found in calculus texts. 


Let b be a positive real number and x and y real numbers. Then 
1. bY = b*b’, and 
2. (GY a", 


We display the graphs of some exponential functions in Figure 1. 


A2.2 Logarithmic Functions 


Suppose that b is a real number with b > 1. Then the exponential function b” is strictly increas- 
ing (a fact shown in calculus). It is a one-to-one correspondence from the set of real numbers 
to the set of nonnegative real numbers. Hence, this function has an inverse log, x, called the 
logarithmic function to the base b. In other words, if b is a real number greater than 1 and x 
is a positive real number, then 


pont = x, 


The value of this function at x is called the logarithm of x to the base b. 
From the definition, it follows that 


log,b* = x. 


We give several important properties of logarithms in Theorem 2. 
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FIGURE 1 Graphs of the exponential functions to the bases * 2, and 5. 


THEOREM 2 — Let bbe a real number greater than 1. Then 


1. log, (xy) = log, x + log, y whenever x and y are positive real numbers, and 
2. log?) = y log, x whenever x is a positive real number and y is a real number. 


Proof: Because log, (xy) is the unique real number with b2% = xy, to prove part 1 it suffices 
to show that b!°&*+!085” = xy, By part 1 of Theorem 1, we have 


plogi xtlog,y — plog, xplog, y 
= xy. 
To prove part 2, it suffices to show that bY!°8»* = x’. By part 2 of Theorem 1, we have 
plg = (ple, xyy 


=x’, < 


The following theorem relates logarithms to two different bases. 


THEOREM 3 CHANGE OF BASE FORMULA FOR LOGARITHMS Leta and b be real numbers 
greater than 1, and let x be a positive real number. Then 


log, x = log, x/log, a. 


Proof: To prove this result, it suffices to show that 


p'8a ¥ log, aay 


By part 2 of Theorem 1, we have 
pk. X-log,a — (b'°8 alog, x 
= eax 
=x 


This completes the proof. < 
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FIGURE 2. The graph of f(x) = log x. 


Because the base used most often for logarithms in this text is b = 2, the notation log x is 
used throughout the test to denote log, x. 

The graph of the function f(x) = log x is displayed in Figure 2. From Theorem 3, when a 
base b other than 2 is used, a function that is a constant multiple of the function log x, namely, 
(1/ log b) log x, is obtained. 


Exercises 

1. Express each of the following quantities as powers of 2. Es" 4, Let a, b, and c be positive real numbers. Show that a! ° = 
a) 2-2 b) (223 c) 22 oe 

2. Find each of the following quantities. 5. Draw the graph of f(x) = b* for all real numbers x if b is 
a) log, 1024 b) log, 1/4 c) log, 8 a) 3. b) 1/3. c) l. 

3. Suppose that log, x = y, where x is a positive real number. 6. Draw the graph of f(x) = log, x for positive real numbers 
Find each of the following quantities. x if bis 


a) log, x b) log, x c) logigx a) 4. b) 100. c) 1000. 
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Pseudocode 


T he algorithms in this text are described both in English and in pseudocode. Pseudocode is 
an intermediate step between an English language description of the steps of a procedure 
and a specification of this procedure using an actual programming language. The advantages 
of using pseudocode include the simplicity with which it can be written and understood and 
the ease of producing actual computer code (in a variety of programming languages) from the 
pseudocode. We will describe the particular types of statements, or high-level instructions, of 
the pseudocode that we will use. Each of these statements in pseudocode can be translated into 
one or more statements in a particular programming language, which in turn can be translated 
into one or more (possibly many) low-level instructions for a computer. 

This appendix describes the format and syntax of the pseudocode used in the text. This 
pseudocode is designed so that its basic structure resembles that of commonly used program- 
ming languages, such as C++ and Java, which are currently the most commonly taught pro- 
gramming languages. However, the pseudocode we use will be a lot looser than a formal 
programming language because a lot of English language descriptions of steps will be allowed. 

This appendix is not meant for formal study. Rather, it should serve as a reference guide for 
students when they study the descriptions of algorithms given in the text and when they write 
pseudocode solutions to exercises. 


A3.1 Procedure Statements 


The pseudocode for an algorithm begins with a procedure statement that gives the name of 
an algorithm, lists the input variables, and describes what kind of variable each input is. For 
instance, the statement 


procedure maximum(L: list of integers) 


is the first statement in the pseudocode description of the algorithm, which we have named 
maximum, that finds the maximum of a list L of integers. 


A3.2 Assignments and Other Types of Statements 


An assignment statement is used to assign values to variables. In an assignment statement the 
left-hand side is the name of the variable and the right-hand side is an expression that involves 
constants, variables that have been assigned values, or functions defined by procedures. The 
right-hand side may contain any of the usual arithmetic operations. However, in the pseudocode 
in this book it may include any well-defined operation, even if this operation can be carried out 
only by using a large number of statements in an actual programming language. 

The symbol := is used for assignments. Thus, an assignment statement has the form 


variable := expression 
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For example, the statement 


assigns the value of a to the variable max. A statement such as 
x := largest integer in the list L 


can also be used. This sets x equal to the largest integer in the list L. To translate this state- 
ment into an actual programming language would require more than one statement. Also, the 
instruction 


interchange a and b 


can be used to interchange a and b. We could also express this one statement with several 
assignment statements (see Exercise 2), but for simplicity, we will often prefer this abbreviated 
form of pseudocode. 


A3.3 Comments 


In the pseudocode in this book, statements enclosed in curly braces are not executed. Such 
statements serve as comments or reminders that help explain how the procedure works. For 
instance, the statement 


{x is the largest element in L} 


can be used to remind the reader that at that point in the procedure the variable x equals the 
largest element in the list L. 


A3.4 Conditional Constructions 


The simplest form of the conditional construction that we will use is 


if condition then statement 


or 


if condition then 
block of statements 
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Here, the condition is checked, and if it is true, then the statement or block of statements given 
is carried out. In particular, the pseudocode 


if condition then 
statement 1 
statement 2 
statement 3 


statement n 


tells us that the statements in the block are executed sequentially if the condition is true. 

For example, in Algorithm 1 in Section 3.1, which finds the maximum of a set of integers, 
we use a conditional statement to check whether max < a; for each variable; if it is, we assign 
the value of a; to max. 

Often, we require the use of a more general type of construction. This is used when we wish 
to do one thing when the indicated condition is true, but another when it is false. We use the 
construction 


if condition then statement 1 
else statement 2 


Note that either one or both of statement 1 and statement 2 can be replaced with a block of 
statements. 

Sometimes, we require the use of an even more general form of a conditional. The general 
form of the conditional construction that we will use is 


if condition 1 then statement 1 
else if condition 2 then statement 2 
else if condition 3 then statement 3 


else if condition n then statement n 
else statement n + 1 


When this construction is used, if condition 1 is true, then statement 1 is carried out, and the 
program exits this construction. In addition, if condition 1 is false, the program checks whether 
condition 2 is true; if it is, statement 2 is carried out, and so on. Thus, if none of the first n — 1 
conditions hold, but condition n does, statement n is carried out. Finally, if none of condition 1, 
condition 2, condition 3, ..., condition n is true, then statement n + 1 is executed. Note that any 
of the n + 1 statements can be replaced by a block of statements. 
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A3.5 Loop Constructions 


There are two types of loop construction in the pseudocode in this book. The first is the “for” 
construction, which has the form 


for variable := initial value to final value 
statement 


or 


for variable := initial value to final value 
block of statements 


where initial value and final value are integers. Here, at the start of the loop, variable is assigned 
initial value if initial value is less than or equal to final value, and the statements at the end of this 
construction are carried out with this value of variable. Then variable is increased by one, and 
the statement, or the statements in the block, are carried out with this new value of variable. 
This is repeated until variable reaches final value. After the instructions are carried out with 
variable equal to final value, the algorithm proceeds to the next statement. When initial value 
exceeds final value, none of the statements in the loop is executed. 

We can use the “for” loop construction to find the sum of the positive integers from 1 to n 
with the following pseudocode. 


sum :=0 
fori :=1ton 
sum := sum +i 


Also, the more general “for” statement, of the form 
for all elements with a certain property 


is used in this text. This means that the statement or block of statements that follow are carried 
out successively for the elements with the given property. 

The second type of loop construction that we will use is the “while” construction. This has 
the form 


while condition 
statement 


or 


while condition 
block of statements 


When this construction is used, the condition given is checked, and if it is true, the statements 
that follow are carried out, which may change the values of the variables that are part of the 
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condition. If the condition is still true after these instructions have been carried out, the instruc- 
tions are carried out again. This is repeated until the condition becomes false. As an example, we 
can find the sum of the integers from 1 to n using the following block of pseudocode including 
a “while” construction. 


sum := 0 
while n > 0 
sum := sum +n 
ne =n E 


Note that any “for” construction can be turned into a “while” construction (see Exercise 3). 
However, it is often easier to understand the “for” construction. So, when it makes sense, we 
will use the “for” construction in preference to the corresponding “while” construction. 
A3.6 Loops within Loops 

Loops or conditional statements are often used within other loops or conditional statements. 


In the pseudocode used in this book, we use successive levels of indentation to indicate nested 
loops, which are loops within loops, and which blocks of commands correspond to which loops. 


A3.7 Using Procedures in Other Procedures 


We can use a procedure from within another procedure (or within itself in a recursive program) 
simply by writing the name of this procedure followed by the inputs to this procedure. For 
instance, 


max(L) 


will carry out the procedure max with the input list L. After all the steps of this procedure have 
been carried out, execution carries on with the next statement in the procedure. 


A3.8 Return Statements 


We use a return statement to show where a procedure produces output. A return statement of 
the form 


return x 


produces the current value of x as output. The output x can involve the value of one or more 
functions, including the same function under evaluation, but at a smaller value. For instance, 
the statement 


return f(n — 1) 


is used to call the algorithm with input of n — 1. This means that the algorithm is run again with 
input equal to n — 1. 
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Exercises 
1. What is the difference between the following blocks of two 2. Give a procedure using assignment statements to inter- 
assignment statements? change the values of the variables x and y. What is the 
minimum number of assignment statements needed to do 
a:i=b this? 
bese 3. Show how a loop of the form 
d for i := initial value to final value 
statement 
b:=c 


a:=b can be written using the “while” construction. 


Suggested Reading 


A mong the resources available for learning more about the topics covered in this book are printed materials 
and relevant websites. Printed resources, including those referred to in the text, are described in this 
section of suggested readings. Readings are listed by chapter and keyed to particular topics of interest. Some 
general references are also provided. A book you may find particularly useful is the Handbook of Discrete and 
Combinatorial Mathematics by Rosen [Ro18], a comprehensive reference book for the material in this book 
and many other topics. Additional applications of discrete mathematics can be found in Michaels and Rosen 
[MiRo91], which is also available online on the companion website for this book. Deeper coverage of many 
topics in computer science, including those discussed in this book, can be found in Gruska [Gr97]. Biographical 
information about many of the mathematicians and computer scientists mentioned in this book can be found in 
Gillispie [Gi70] and on the MacTutor website at /itip://www-history.mcs.st-and.ac.uk/. 


To find pertinent websites, consult the links found in the Web Resources Guide on the companion website 
for this book. Its address is www.mhhe.com/rosen. 


CHAPTER 1 


An entertaining way to study logic is to read Lewis Carroll’s book 
[Ca78]. General references for logic include M. Huth and M. Ryan 
[HuRy04], Mendelson [Me09], Stoll [St74], and Suppes [Su87]. A 
comprehensive treatment of logic in discrete mathematics can be 
found in Gries and Schneider [GrSc93]. System specifications are 
discussed in Ince [In93]. Smullyan’s knights and knaves puzzles 
were introduced in [Sm78]. He wrote many fascinating books on 
logic puzzles including [Sm92] and [Sm98]. Prolog is discussed 
in depth in Nilsson and Maluszynski [NiMa95] and in Clocksin 
and Mellish [ClMe94]. The basics of proofs are covered in 
Cupillari [Cu05], Morash [Mo91], Solow [S009], Velleman 
[Ve06], and Wolf [Wo98]. The science and art of constructing 
proofs is discussed in a delightful way in three books by Pólya: 
[P062], [Po71], and [Po90]. Problems involving tiling checker- 
boards using dominoes and polyominoes are discussed in Golomb 
[Go94] and Martin [Ma91]. 


CHAPTER 2 


Lin and Lin [LiLi81] is an easily read text on sets and their ap- 
plications. Axiomatic developments of set theory can be found in 
Halmos [Ha60], Monk [Mo69], Pinter [Pil4], and Stoll [St74]. 
Brualdi [Br09], and Reingold, Nievergelt, and Deo [ReNiDe77] 
contain introductions to multisets. Fuzzy sets and their application 
to expert systems and artificial intelligence are treated in Negoita 
[Ne85] and Zimmerman [Zi91]. Calculus books, such as Apostol 
[Ap67], Spivak [Sp94], and Thomas and Finney [ThFi96], contain 
discussions of functions. The best printed source of information 
about integer sequences is Sloan and Plouffe [SIP195]. Books on 
proofs, such as [Ve06], often cover countability in some depth. 
Stanat and McAllister [StMc77] has a thorough section on count- 
ability. Chapter 17 of Aigner and Ziegler [AiZi14] provides an 
excellent discussion of cardinality and the continuum hypothesis. 


Discussions of the mathematical foundations needed for computer 
science can be found in Arbib, Kfoury, and Moll [ArKfMo80]; 
Bobrow and Arbib [BoAr74]; Beckman [Be80]; and Tremblay 
and Manohar [TrMa75]. Matrices and their operations are cov- 
ered in all linear algebra books, such as Curtis [Cu84] and Strang 
[St09]. 


CHAPTER 3 


The articles by Knuth [Kn77] and Wirth [Wi84] are accessi- 
ble introductions to the subject of algorithms. Among the best 
introductions to algorithms are Cormen, Leierson, Rivest, and 
Stein [CoLeRiSt09] and Kleinberg and Tardos [KITa05]. Ex- 
tensive material on big-O estimates of functions can be found 
in Knuth [Kn97a]. General references for algorithms and their 
complexity include Aho, Hopcroft, and Ullman [AhHoUI74]; 
Baase and Van Gelder [BaGe99]; Cormen, Leierson, Rivest, 
and Stein [CoLeRiSt09]; Gonnet [Go84]; Goodman and Hedet- 
niemi [GoHe77]; Harel [Ha87]; Horowitz and Sahni [HoSa82]; 
Kreher and Stinson [KrSt98]; the famous series of books by 
Knuth on the art of computer programming [Kn97a], [Kn97b], 
and [Kn98]; Kronsjé [Kr87]; Levitin [Le06]; Manber [Ma89]; 
Pohl and Shaw [PoSh81]; Purdom and Brown [PuBr85]; Rawlins 
[Ra92]; Sedgewick [Se03]; Wilf [Wi02]; and Wirth [W176]. Sort- 
ing and searching algorithms and their complexity are studied in 
detail in Knuth [Kn98]. 


CHAPTER 4 


References for number theory include Hardy and Wright [HaWr- 
WiHe08], LeVeque [Le77], Rosen [Ro10], and Stark [St78]. More 
about the history of number theory can be found in Ore [Or88]. Al- 
gorithms for computer arithmetic are discussed in Knuth [Kn97b] 
and Pohl and Shaw [PoSh81]. More information about algorithms 
for finding primes and for factorization can be found in Cran- 
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dall and Pomerance [CrPo10]. Applications of number theory to 
cryptography are covered in Denning [De82]; Menezes, van 
Oorschot, and Vanstone [MeOoVa97]; Rosen [Ro10]; Seberry and 
Pieprzyk [SePi89]; Sinkov [S166]; and Stinson [St05]. The RSA 
public-key system was described by Rivest, Shamir, and Adle- 
man in [RiShAd78]; its discovery by Cocks is described in [Si99], 
which also provides an appealing account of the history of cryp- 
tography. 


CHAPTER 5 


An accessible introduction to mathematical induction can be 
found in [Gu10] and in Sominskii [S061]. Books that contain 
thorough treatments of mathematical induction and recursive def- 
initions include Liu [Li85], Sahni [Sa85], Stanat and McAllister 
[StMc77], and Tremblay and Manohar [TrMa75]. Computational 
geometry is covered in [DeOr11] and [Or00]. The Ackermann 
function, introduced in 1928 by W. Ackermann, arises in the the- 
ory of recursive function (see Beckman [Be80] and McNaughton 
[Mc82], for instance) and in the analysis of the complexity of 
certain set theoretic algorithms (see Tarjan [Ta83]). Recursion is 
studied in Roberts [Ro86], Rohl [Ro84], and Wand [Wa80]. Dis- 
cussions of program correctness and the logical machinery used to 
prove that programs are correct can be found in Alagic and Arbib 
[AlAr78], Anderson [An79], Backhouse [Ba86], Sahni [Sa85], 
and Stanat and McAllister [StMc77]. 


CHAPTER 6 


General references for counting techniques and their applica- 
tions include Allenby and Slomson [AIS110]; Anderson [An89]; 
Berman and Fryer [BeFr72]; Bogart [Bo00]; Bona [Bo07]; Bose 
and Manvel [BoMa86]; Brualdi [Br09]; Cohen [C078]; Grimaldi 
[Gr03]; Gross [Gr07]; Liu [Li68]; Pólya, Tarjan, and Woods 
[PoTaWo83]; Riordan [Ri58]; Roberts and Tesman [RoTe03]; 
Tucker [Tu06]; and Williamson [Wi85]. Vilenkin [Vi71] contains 
a selection of combinatorial problems and their solutions. A se- 
lection of more difficult combinatorial problems can be found 
in Lovasz [Lo79]. Information about Internet protocol addresses 
and datagrams can be found in Comer [Co05]. Applications of 
the pigeonhole principle can be found in Brualdi [Br09], Liu 
[Li85], and Roberts and Tesman [RoTe03]. A wide selection of 
combinatorial identities can be found in Riordan [Ri68] and in 
Benjamin and Quinn [BeQu03]. Combinatorial algorithms, in- 
cluding algorithms for generating permutations and combinations, 
are described by Even [Ev73]; Lehmer [Le64]; and Reingold, 
Nievergelt, and Deo [ReNiDe77]. 


CHAPTER 7 


Useful references for discrete probability theory include Feller 
[Fe68], Nabin [Na00], and Ross [Ro09a]. Ross [Ro02], which fo- 
cuses on the application of probability theory to computer science, 
provides examples of average case complexity analysis and cov- 
ers the probabilistic method. Aho and Ullman [AhU195] includes 
a discussion of various aspects of probability theory important in 


computer science, including programming applications of proba- 
bility. The probabilistic method is discussed in a chapter in Aigner 
and Ziegler [AiZi14], a monograph devoted to clever, insightful, 
and brilliant proofs, that is, proofs that Paul Erdős described as 
coming from “The Book.” Extensive coverage of the probabilistic 
method can be found in Alon and Spencer [A]Sp00]. Bayes’ the- 
orem is covered in [PaPi01]. Additional material on spam filters 
can be found in [Zd05]. 


CHAPTER 8 


Many different models using recurrence relations can be found 
in Roberts and Tesman [RoTe03] and Tucker [Tu06]. Exhaus- 
tive treatments of linear homogeneous recurrence relations with 
constant coefficients, and related inhomogeneous recurrence re- 
lations, can be found in Brualdi [Br09], Liu [Li68], and Mattson 
[Ma93]. Divide-and-conquer algorithms and their complexity are 
covered in Roberts and Tesman [RoTe03] and Stanat and McAl- 
lister [StMc77]. Descriptions of fast multiplication of integers and 
matrices can be found in Aho, Hopcroft, and Ullman [AhHoUI174] 
and Knuth [Kn97b]. An excellent introduction to generating func- 
tions can be found in Polya, Tarjan, and Woods [PoTaWo83]. Gen- 
erating functions are studied in detail in Brualdi [Br09]; Cohen 
[C078]; Graham, Knuth, and Patashnik [GrknPa94]; Grimaldi 
[Gr03]; and Roberts and Tesman [RoTe03]. Additional applica- 
tions of the principle of inclusion—exclusion can be found in 
Liu [Li85] and [Li68], Roberts and Tesman [RoTe03], and Ryser 
[Ry63]. 


CHAPTER 9 


General references for relations, including treatments of equiv- 
alence relations and partial orders, include Bobrow and Ar- 
bib [BoAr74], Grimaldi [Gr03], Sanhi [Sa85], and Tremblay 
and Manohar [TrMa75]. Discussions of relational models for 
databases are given in Date [Da82] and Aho and Ullman 
[AhU195]. The original papers by Roy and Warshall for find- 
ing transitive closures can be found in [Ro59] and [Wa62], re- 
spectively. Directed graphs are studied in Chartrand, Lesniak, 
and Zhang [ChLeZh15]; Gross and Yellen [GrYe05]; Robinson 
and Foulds [RoFo80]; Roberts and Tesman [RoTe03]; and Tucker 
[Tu06]. The application of lattices to information flow is treated 
in Denning [De82]. 


CHAPTER 10 


General references for graph theory include Agnarsson and 
Greenlaw [AgGr06]; Aldous, Wilson, and Best [AlWiBe00]; Be- 
hzad and Chartrand [BeCh71]; Chartrand, Lesniak, and Zhang 
([ChLeZh15]; Chartrand and Zhang [ChZh04]; Bondy and Murty 
[BoMul0]; Chartrand and Oellermann [ChOe93]; Graver and 
Watkins [GrWa77]; Roberts and Tesman [RoTe03]; Tucker 
[Tu06]; West [We00]; Wilson [Wi85]; and Wilson and Watkins 
[WiWa90]. A wide variety of applications of graph theory can be 
found in Chartrand [Ch77], Deo [De74], Foulds [Fo92], Roberts 
and Tesman [RoTe03], Roberts [Ro76], Wilson and Beineke 


[WiBe79], and McHugh [Mc90]. In-depth treatments of the use 
of graph theory to study social networks, and other types of 
networks, appears in Easley and Kleinberg [EaK110] and New- 
man [Ne10]. Applications involving large graphs, including the 
Web graph, are discussed in Hayes [Ha00a] and [Ha00b]. 

A comprehensive description of algorithms in graph the- 
ory can be found in Gibbons [Gi85] and in Kocay and Kre- 
her [KoKr04]. Other references for algorithms in graph theory 
include Buckley and Harary [BuHa90]; Chartrand and Oeller- 
mann [ChOe93]; Chachra, Ghare, and Moore [ChGhMo79]; Even 
[Ev73] and [Ev79]; Hu [Hu82]; and Reingold, Nievergelt, and 
Deo [ReNiDe77]. A translation of Euler’s original paper on 
the Königsberg bridge problem can be found in Euler [Eu53]. 
Dikstra’s algorithm is studied in Gibbons [Gi85]; Liu [Li85]; and 
Reingold, Nievergelt, and Deo [ReNiDe77]. Dijkstra’s original 
paper can be found in [Di59]. A proof of Kuratowski’s theorem 
can be found in Harary [Ha69] and Liu [L168]. Crossing num- 
bers and thicknesses of graphs are studied in Chartrand, Lesniak, 
and Zhang [ChLeZh15]. References for graph coloring and the 
four-color theorem are included in Barnette [Ba83] and Saaty and 
Kainen [SaKa86]. The original conquest of the four-color theo- 
rem is reported in Appel and Haken [ApHa76]. Applications of 
graph coloring are described by Roberts and Tesman [RoTe03]. 
The history of graph theory is covered in Biggs, Lloyd, and Wil- 
son [BiL]Wi86]. Interconnection networks for parallel processing 
are discussed in Akl [Ak89] and Siegel and Hsu [SiHs88]. 


CHAPTER 11 


Trees are studied in Deo [De74], Grimaldi [Gr03], Knuth 
[Kn97a], Roberts and Tesman [RoTe03], and Tucker [Tu06]. 
The use of trees in computer science is described by Gotlieb 
and Gotlieb [GoGo78], Horowitz and Sahni [HoSa82], and 
Knuth [Kn97a] and [Kn98]. Roberts and Tesman [RoTe03] cov- 
ers applications of trees to many different areas. Combina- 
torial game theory is studied in [AINoWo07], [BeCoGu01], 
and [Bell]. Prefix codes and Huffman coding are covered in 
Hamming [Ha80]. Backtracking is an old technique; its use to 
solve maze puzzles can be found in the 1891 book by Lucas 
[Lu91]. An extensive discussion of how to solve problems us- 
ing backtracking can be found in Reingold, Nievergelt, and Deo 
[ReNiDe77]. Gibbons [Gi85] and Reingold, Nievergelt, and 
Deo [ReNiDe77] contain discussions of algorithms for construct- 
ing spanning trees and minimal spanning trees. The background 
and history of algorithms for finding minimal spanning trees is 
covered in Graham and Hell [GrHe85]. Prim and Kruskal de- 
scribed their algorithms for finding minimal spanning trees in 
[Pr57] and [Kr56], respectively. Sollin’s algorithm is an example 
of an algorithm well suited for parallel processing; although Sollin 
never published a description of it, his algorithm has been de- 
scribed by Even [Ev73] and Goodman and Hedetniemi [GoHe77]. 


CHAPTER 12 


Boolean algebra is studied in Hohn [Ho66], Kohavi [Ko86], and 
Tremblay and Manohar [TrMa75]. Applications of Boolean alge- 
bra to logic circuits and switching circuits are described by Hayes 
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[Ha93], Hohn [Ho66], Katz and Borriello [KaBo04], and Ko- 
havi [Ko86]. The original papers dealing with the minimization 
of sum-of-products expansions using maps are Karnaugh [Ka53] 
and Veitch [Ve52]. The Quine-McCluskey method was introduced 
in McCluskey [Mc56] and Quine [Qu52] and [Qu55]. Threshold 
functions are covered in Kohavi [Ko86]. 


CHAPTER 13 


General references for formal grammars, automata theory, and 
the theory of computation include Davis, Sigal, and Weyuker 
[DaSiWe94]; Denning, Dennis, and Qualitz [DeDeQu81]; 
Hopcroft, Motwani, and Ullman [HoMoUI106]; Hopkin and Moss 
[HoMo76]; Lewis and Papadimitriou [LePa97]; McNaughton 
[Mc82]; and Sipser [S106]. Mealy machines and Moore machines 
were originally introduced in Mealy [Me55] and Moore [M056]. 
The original proof of Kleene’s theorem can be found in [K156]. 
Powerful models of computation, including pushdown automata 
and Turing machines, are discussed in Brookshear [Br89], Hen- 
nie [He77], Hopcroft and Ullman [HoU179], Hopkin and Moss 
[HoMo76], Martin [Ma03], Sipser [Si06], and Wood [W087]. 
Barwise and Etchemendy [BaEt93] is an excellent introduction 
to Turing machines. Interesting articles about the history and ap- 
plication of Turing machines and related machines can be found 
in Herken [He88]. Busy beaver machines were first introduced 
by Rado in [Ra62], and information about them can be found 
in Dewdney [De84] and [De93], the article by Brady in Herken 
[He88], and in Wood [W087]. 


APPENDIXES 


A discussion of axioms for the real number and for the integers 
can be found in Morash [Mo91]. Detailed treatments of expo- 
nential and logarithmic functions can be found in calculus books 
such as Apostol [Ap67], Spivak [Sp94], and Thomas and Finney 
[ThFi96]. Pohl and Shaw [PoSh81] use a form of pseudocode that 
has the same features as those described in Appendix 3. Most text- 
books on algorithms, such as Cormen, Leierson, Rivest, and Stein 
[CoLeRiSt09] and Kleinberg and Tardos [KITa05], use versions 
of pseudocode similar to the pseudocode in this text. 
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Answers to Odd-Numbered Exercises 


CHAPTER 1 
Section 1.1 
1. a) Yes, Tb) Yes, Fc) Yes, T d) Yes, F e)No 


f) No 3. a) Linda is not younger than Sanjay. b) Mei does 
not make more money than Isabella. c) Moshe is not taller 
than Monica. d) Abby is not richer than Ricardo. 5. a) Mei 
does not have an MP3 player. b) There is pollution in New 
Jersey. c)2+1 43. d) The summer in Maine is not hot or 
it is not sunny. 7. a) Steve does not have more than 100 
GB free disk space on his laptop. b) Zach does not block e- 
mails from Jennifer, or he does not block texts from Jennifer. 
c) 7-11-13 4999. d) Diane did not ride her bike 100 miles 
on Sunday. 9%. a)F b)T c)T d)T e)T 11. a) Sharks 
have not been spotted near the shore. b) Swimming at the 
New Jersey shore is allowed, and sharks have been spotted 
near the shore. c) Swimming at the New Jersey shore is not 
allowed, or sharks have been spotted near the shore. d) If 
swimming at the New Jersey shore is allowed, then sharks 
have not been spotted near the shore. e) If sharks have not 
been spotted near the shore, then swimming at the New Jersey 
shore is allowed. f) If swimming at the New Jersey shore is 
not allowed, then sharks have not been spotted near the shore. 
g) Swimming at the New Jersey shore is allowed if and only 
if sharks have not been spotted near the shore. h) Swimming 
at the New Jersey shore is not allowed, and either swimming 
at the New Jersey shore is allowed or sharks have not been 
spotted near the shore. (Note that we were able to incorpo- 
rate the parentheses by using the word “either” in the second 
half of the sentence.) I3.apAq b)pA-7q e¢)7pAr7q 
Dpvq pq hPvgAw-g) g)qep 15.a)7p 
bpr op>q dps q ep->q qap 
gq->p 17.arA7p b)7ApAgAr ©r > (q e p) 
d) agampar e)(q>CrAnp))AACranp) > q) 
f)(pAr) > ~q 19.a) False b)True c¢)True d) True 
21. a) Exclusive or: You get only one beverage. b) Inclusive 
or: Long passwords can have any combination of symbols. 
c) Inclusive or: A student with both courses is even more qual- 
ified. d) Either interpretation possible; a traveler might wish 
to pay with a mixture of the two currencies, or the store may 
not allow that. 23. a) Inclusive or: It is allowable to take 
discrete mathematics if you have had calculus or computer 
science, or both. Exclusive or: It is allowable to take discrete 
mathematics if you have had calculus or computer science, 
but not if you have had both. Most likely the inclusive or is 
intended. b) Inclusive or: You can take the rebate, or you can 
get a low-interest loan, or you can get both the rebate and a 
low-interest loan. Exclusive or: You can take the rebate, or 
you can get a low-interest loan, but you cannot get both the 
rebate and a low-interest loan. Most likely the exclusive or is 
intended. c) Inclusive or: You can order two items from col- 


umn A and none from column B, or three items from column 
B and none from column A, or five items including two from 
column A and three from column B. Exclusive or: You can 
order two items from column A or three items from column 
B, but not both. Almost certainly the exclusive or is intended. 
d) Inclusive or: More than 2 feet of snow or windchill below 
—100 °F, or both, will close school. Exclusive or: More than 
2 feet of snow or windchill below —100 °F, but not both, will 
close school. Certainly the inclusive or is intended. 25. a) If 
the wind blows from the northeast, then it snows. b) If it 
stays warm for a week, then the apple trees will bloom. e) If 
the Pistons win the championship, then they beat the Lakers. 
d) If you get to the top of Long’s Peak, then you must have 
walked 8 miles. e) If you are world famous, then you will get 
tenure as a professor. f) If you drive more than 400 miles, 
then you will need to buy gasoline. g) If your guarantee is 
good, then you must have bought your CD player less than 
90 days ago. h) If the water is not too cold, then Jan will go 
swimming. i) If people believe in science, then we will have a 
future. 27. a) You buy an ice cream cone if and only if it is 
hot outside. b) You win the contest if and only if you hold the 
only winning ticket. c) You get promoted if and only if you 
have connections. d) Your mind will decay if and only if you 
watch television. e) The train runs late if and only if it is a day 
I take the train. 29. a) Converse: “I will ski tomorrow only 
if it snows today.” Contrapositive: “If I do not ski tomorrow, 
then it will not have snowed today.” Inverse: “If it does not 
snow today, then I will not ski tomorrow.” b) Converse: “If 
I come to class, then there will be a quiz.” Contrapositive: “If 
I do not come to class, then there will not be a quiz.” Inverse: 
“Tf there is not going to be a quiz, then I don’t come to class.” 
c) Converse: “A positive integer is a prime if it has no divisors 
other than 1 and itself.” Contrapositive: “If a positive integer 
has a divisor other than 1 and itself, then it is not prime.” In- 


verse: “If a positive integer is not prime, then it has a divisor 
other than 1 and itself.” 31. a)2 b)16 c)64 d)16 


S-2 Answers to Odd-Numbered Exercises 


izo iieo iiae iae i lin 


Mia's 


zo ies ieo ies ia Bar Har Har i iin 
goie iir Har ieo ieo Ber Har i ian 


37. 


Midas 


39. 


zo ies ieo ies ia Bar far Bar i iin 


TH iieo iar i la>) 


TmAHA TM THH)sS 


p> (qvr) 
T 


(pVq) > (PAQq) 


smd} 


=p > (q >r) 


T T T T 
T T T T 
T T F T 
T T F F 
T T T F 
F T F T 
T T T T 
T T F T 


e) 


THAS 


(p>q)V 
(=p >r) 


(p> q)A 
(=p >r) 


(=p e ~q) 


(pq) 


HHHH 


(poqgyv 
q er) 


(p >44) © 
(~q > =p) 


EEEE 


(p ® q) > (p ® ~q) 


T 
F 
F 
T 
(p e ~q) > 
qer) 

T 

F 

T 

F 

F 

T 

F 

T 


41. 


TIT TMT TDAH AAAS Har i ir 
TUT TMHAHAA TIT TAAYAHIsS 
TMAH TM TAAT THAT THAH) ia 
MATH TMHATMATH TMH TA THe 
Ge ies iiz i fa Daan DTH aH! 


T 
T 
T 
T 
F 
F 
F 
F 
F 
F 
F 
F 
T 
T 
T 
T 


Sesle. io ier er Mee ieo ies k io es ie irs ies i 


43. The first clause is true if and only if at least one of p, q, and 
r is true. The second clause is true if and only if at least one 
of the three variables is false. Therefore, the entire statement 
is true if and only if there is at least one T and one F among the 
truth values of the variables, pe other words, that they don’t all 
have the same truth value. 45. (A) Nizi OPi V =p;))) A 
(Vii, Pi) 47. a) Bitwise Be is 111 1111; bitwise AND is 
000 0000; bitwise XOR is 111 1111. b) Bitwise OR is 1111 
1010; bitwise AND is 1010 0000; bitwise XOR is 0101 1010. 
c) Bitwise OR is 10 0111 1001; bitwise AND is 00 0100 0000; 
bitwise XOR is 100011 1001. d) Bitwise OR is 11 1111 1111; 
bitwise AND is 00 0000 0000; bitwise XOR is 11 1111 1111. 
49. 0.2,0.6 51. 0.8,0.6 53. a) The 99th statement is true 
and the rest are false. b) Statements | through 50 are all true 
and statements 51 through 100 are all false. c) This cannot 
happen; it is a paradox, showing that these cannot be state- 
ments. 
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l.e>a 3.g3(rACAM)A(CAD)) 5.e > (AA(bVp)Ar) 
7.ajq>p b)q A =p &q>p d-~q>r7p 9. Not 
consistent 11. Consistent 13. NEW AND JERSEY 
AND BEACHES, (JERSEY AND BEACHES) NOT NEW 
15. “ETHIOPIAN RESTAURANTS” AND (“NEW YORK” 
OR “NEW JERSEY”) 17. a) Queen cannot say this. 
b) Queen can say this, but one cannot determine location 
of treasure. €c) Queen can say this; treasure is in Trunk 1. 
d) Queen cannot say this. 19. “If I were to ask you whether 
the right branch leads to the ruins, would you answer yes?” 
21. If the first professor did not want coffee, then he would 
know that the answer to the hostess’s question was “no.” 
Therefore the hostess and the remaining professors know 
that the first professor did want coffee. Similarly, the sec- 
ond professor must want coffee. When the third professor 
said “no,” the hostess knows that the third professor does 
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not want coffee. 23. Ais a knight and B is a knave. 25. 
A is a knight and B is a knight. 27. A is a knave and B 
is a knight. 29. A is the knight, B is the spy, C is the 
knave. 31. A is the knight, B is the spy, C is the knave. 
33. Any of the three can be the knight, any can be the 
spy, any can be the knave. 35. No solutions 37. In or- 
der of decreasing salary: Fred, Maggie, Janice 39. The 
detective can determine that the butler and cook are lying 
but cannot determine whether the gardener is telling the 
truth or whether the handyman is telling the truth. 41. The 
Japanese man owns the zebra, and the Norwegian drinks wa- 
ter. 43. One honest, 49 corrupt 45. a) 7=(p A (q V 77)) 
b) (=D) ACM) V EAT) 
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1. The equivalences follow by showing that the appropriate 
pairs of columns of this table agree. 
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7.a) Jan is not rich, or Jan is not happy. b) Carlos will not bi- 
cycle tomorrow, and Carlos will not run tomorrow. c) Mei 
does not walk to class, and Mei does not take the bus to 
class. d) Ibrahim is not smart, or Ibrahim is not hard working. 
9.a) ap V =q b) (paq) yvr ¢) pV =q 
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11. a) 


13. a) If this were not a tautology, then p A q would be true 
but p would be false. This cannot happen, because the truth of 
p Aq implies the truth of p. b) If this were not a tautology, 
then p would be true but p V q would be false. This cannot 
happen, because the truth of p implies the truth of pV q. c) If 
this were not a tautology, then p would be false and p > q 
would be false. This cannot happen, because p — q is true 
when p is false. d) If this were not a tautology, then p A q 
would be true and p — q would be false. This cannot happen, 
because p — q is true when both p and q are true. e) If 
this were not a tautology, then p — q would be false and p 
would be false. This cannot happen, because p — q is true 
when p is false. f) If this were not a tautology, then p > q 
would be false and q would be true. This cannot happen, be- 
cause p > q is true when q is true. 15. a)(pAq) > p= 
“Ap AQVp=—pV-~qV p= (pV p) q=Tv~q4=T 
bp > PVQ=PVEVQ=CPYP)VG=TVG=T 
c) =p > (p> 4) = PV (p > 9) = PVCPVG) = (PV-P)VG= 


Tvq4=T DPAD > PF QD=7PADVCPVOD= 
=pV7qV-=pVg = (pV-p)V(7qVq) =- pVT=T_ e) p> 
gq) > Pp=(P > QVP = —PVGVP=(CpvP)Vqg=Tvqg=T 
f) =P > q) > 74 = (p >4)V q = =pVqV-q=-pvT=T 
17. That the fourth column of the truth table shown is identical 
to the first column proves part (a), and that the sixth column 
is identical to the first column proves part (b). 
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19. It is a tautology. 21. Each of these is true precisely 
when p and q have opposite truth values. 23. The propo- 
sition =p + q is true when 7p and q have the same truth 
values, which means that p and q have different truth val- 
ues. Similarly, p + ~q is true in exactly the same cases. 
Therefore, these two expressions are logically equivalent. 
25. The proposition =(p + q) is true when p © q is false, 
which means that p and q have different truth values. Because 
this is precisely when =p < q is true, the two expressions are 
logically equivalent. 27. For (p > r) A (q > r) to be false, 
one of the two conditional statements must be false, which 
happens exactly when r is false and at least one of p and q is 
true. But these are precisely the cases in which pV q is true and 
r is false, which is precisely when (pV q) > r is false. Because 
the two propositions are false in exactly the same situations, 
they are logically equivalent. 29. For (p > r) V (q > r) to 
be false, both of the two conditional statements must be false, 
which happens exactly when r is false and both p and q are 
true. But this is precisely the case in which p A q is true and r 
is false, which is precisely when (p A q) > r is false. Because 
the two propositions are false in exactly the same situations, 
they are logically equivalent. 31. This fact was observed 
in Section 1 when the biconditional was first defined. Each 
of these is true precisely when p and q have the same truth 
values. 33. The last column is all Ts. 
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35. These are not logically equivalent because when p, q, and 
r are all false, (0 > q) > ris false, but p > (q > r)is 
true. 37. Many answers are possible. If we let r be true and 


p, q, and s be false, then (p > q) > (r — s) will be false, 
but (p > r) > (q > s) will be true. 39. a) p V ^q V >r 
bpVaVnAs OMAT)V(GAF) 4l. If we take 
duals twice, every V changes to an A and then back to an 
V, every A changes to an V and then back to an A, every T 
changes to an F and then back to a T, every F changes to 
a T and then back to an F. Hence, (s*)* = s. 43. Let p 
and q be equivalent compound propositions involving only 
the operators A, V, and ~, and T and F. Note that ap and =q 
are also equivalent. Use De Morgan’s laws as many times as 
necessary to push negations in as far as possible within these 
compound propositions, changing Vs to As, and vice versa, 
and changing Ts to Fs, and vice versa. This shows that =p 
and ~q are the same as p* and q* except that each atomic 
proposition p; within them is replaced by its negation. From 
this we can conclude that p* and q* are equivalent because 
=p and aq are. 45.(pAGA7r)V(pA7qGAr)V paqar) 
47. Given a compound proposition p, form its truth table 
and then write down a proposition q in disjunctive normal 
form that is logically equivalent to p. Because q involves 
only ~=, A, and v, this shows that these three operators form 
a functionally complete set. 49. By Exercise 47, given a 
compound proposition p, we can write down a proposi- 
tion q that is logically equivalent to p and involves only 
=, A, and V. By De Morgan’s law we can eliminate all the 
As by replacing each occurrence of p; Ap, A+ App, with 
aP V 7p. V + Vap,). 51.~(p A q) is true when either 
p or q, or both, are false, and is false when both p and q are 
true. Because this was the definition of p | q, the two com- 
pound propositions are logically equivalent. 53. =~(p V q) 
is true when both p and q are false, and is false otherwise. 
Because this was the definition of p | q, the two are logically 
equivalent. 55.(plp)lq)il(plp)lq) 57. This fol- 
lows immediately from the truth table or definition of p | q. 
59.16 61. If the database is open, then either the system 
is in its initial state or the monitor is put in a closed state. 
63. All nine 65. a) Satisfiable b) Not satisfiable c) Not 


satisfiable 67. a) ( Ai Vi PG D) A ( Aah a a= 
CpGDV—PGD) A (Api Ar Nein PGY 
WED) ® (A Aja A PG) V WE k, 
KHD) A (AA AEE POV plit k, 
J+) ); No solutions possible. b) (Aj, Vin PD) A 
(AnA aA ota t MA A aAa 
PEDDY DED) A (A Aa Me? G) 
“PE k k+ JAA N AE POG) YE +k 
j+%)); No solutions possible. ¢) (Ai, Vj. PGD) A 
(Air Aver Nectar CPG. Y BG ))ACAjer Adar Mesias 
PEDDY PED) A (Nia Aj AO? Cony 
api — kk + D) ^ (An: Na Ne CPG DY 
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—pi+kitb)); 1 D 2&4 G D & 3) oF 
(1, 3), (2, 1), (3, 4), (4, 2) 69. Use the same propositions 
as were given in the text for a 9 x 9 Sudoku puzzle, with 
the variables indexed from 1 to 4, instead of from 1 to 9, 
and with a similar change for the propositions for the 2 x 2 
blocks: Avs New Na V}, Vai p(2r + i, 2s +j n) 
71. V, p(i, j, n) asserts that column j contains the num- 
ber n, so Na Va p(i, j, n) asserts that column j contains 
all 9 numbers; therefore Na Na VŽ p(i, j, n) asserts that 
every column contains every number. 
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l.a)T b)T ©F 3.a)T b)F c)F d) F 5.a)There 
is a student who spends more than 5 hours every weekday 
in class. b) Every student spends more than 5 hours ev- 
ery weekday in class. c) There is a student who does not 
spend more than 5 hours every weekday in class. d) No 
student spends more than 5 hours every weekday in class. 
7. a) Every comedian is funny. b) Every person is a funny 
comedian. c) There exists a person such that if she or he 
is a comedian, then she or he is funny. d) Some comedi- 
ans are funny. 9. a) dx(P(x) A Q(x)) b) Ax(P(x) A 7Q(x)) 
e) Vx(P(x) V O(x)) d) Vx7(P(x) V Q(x) 11.a T b)T oF 
dF eT F 13.a)T bT c)T dT 15.a)T b)F 
c)T d)F 17.a PO) v P) v PQ) v PB) v PA) 
b) P(O) A P(1) A P(2) A PB) A P(A) e) APO) V APC) v 
aP(2) v =P(3) v aP(4) d) =P(O) A aP(1) A AP(2) AAP(3) A 
aP(4) e) 7(PCO)VPC)VP(2)V P(3)V P(4)) £) ~(POO)VAPCL) A 
P(2) A P(3) A P(4)) 19. a) P(1) v P(2) v PB) v P(4) v PS) 
b) P(1) A P(2) A PB) A PA A P(S) e) (P(A) v P) v 
P(3) VP(4) VP(5)) d) a(P(1) A P(2) A P(3) A P(A) A P(S)) 
e) (PU) AP(2) A P(4) A P(S)) V (APC) v aP(2) v aP) v 
aP(4) v =P(5)) 21. Many answers are possible. a) All 
students in your discrete mathematics class; all students in 
the world b) All United States senators; all college football 
players c) George W. Bush and Jeb Bush; all politicians in 
the United States d) Bill Clinton and George W. Bush; all 
politicians in the United States 23. Let C(x) be the proposi- 
tional function “x is in your class.” a) SxH(x) and 3x(C(x) A 
HA(x)), where H(x) is “x can speak Hindi” b) VxF(x) and 
Vx(C(x) > F(x)), where F(x) is “x is friendly” ce) 4x-B(x) 
and Ax(C(x) A7B(x)), where B(x) is “x was born in California” 
d) AxM(x) and Ax(C(x) A M(x)), where M(x) is “x has been in 
a movie” e) Vx7L(x) and Vx(C(x) > 7=L(x)), where L(x) is “x 
has taken a course in logic programming” 25. Let P(x) be 
“x 1s perfect’; let F(x) be “xis your friend”; and let the domain 
be all people. a) Vx =P(x)_b) =Vx P(x) ©) Vx(F(x) > PO) 
d) Ax(F(x) A P(x)) e€) YF (x) A PŒ) or (Wx F(x) A (Wx P@)) 
Ê) Yx F(x) V(Ax 7P(@x)) 27. Let Y(x) be the propositional 
function that x is in your school or class, as appropriate. a) If 
we let V(x) be “x has lived in Vietnam,” then we have 4xV(x) 
if the domain is just your schoolmates, or 4x(Y(x) A V(x)) 
if the domain is all people. If we let D(x, y) mean that per- 
son x has lived in country y, then we can rewrite this last one 
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as dx(Y(x) A D(x, Vietnam)). b) If we let H(x) be “x can 
speak Hindi,” then we have 4x7H(x) if the domain is just your 
schoolmates, or 4x(Y(x) A ~H (x)) if the domain is all people. 
If we let S(x, y) mean that person x can speak language y, then 
we can rewrite this last one as 3x(Y (x) A 7S(x, Hindi)). c) If 
we let J(x), P(x), and C(x) be the propositional functions as- 
serting x’s knowledge of Java, Prolog, and C++, respectively, 
then we have 3x(J(x) A P(x) A C(x)) if the domain is just 
your schoolmates, or 3x(Y (x) A JŒ) A P(x) A C(x)) if the do- 
main is all people. If we let K(x, y) mean that person x knows 
programming language y, then we can rewrite this last one as 
Ax(Y(x) A K(x, Java) A K(x, Prolog) A K(x, C++)). d) If 
we let T(x) be “x enjoys Thai food,” then we have Vx T(x) if 
the domain is just your classmates, or Vx(Y(x) > T(x)) if 
the domain is all people. If we let E(x, y) mean that person 
x enjoys food of type y, then we can rewrite this last one as 
Vx(¥(x) > E(x, Thai)). e) If we let H(x) be “x plays hockey,” 
then we have 4x ~H (x) if the domain is just your classmates, 
or 3x(Y(x)A7A(x)) if the domain is all people. If we let P(x, y) 
mean that person x plays game y, then we can rewrite this last 
one as 3x(Y (x) A7P(x, hockey)). 29. Let T(x) mean that x is 
a tautology and C(x) mean that x is a contradiction. a) 3x T(x) 
b) Vx(C(x) > T(>x)) c) daxdSy(AT(x) A aC) A aT) A 
SCAT V y)) DWT) ATO) > TAN) 
31. a) Q(0, 0, 0) a Q(0, 1,0) b)Q(0,1,1)v Q0, 1,1) v 
Q(2, 1,1) c) ~Q(0, 0, 0) v ~Q(0, 0, 1) d)~Q(0, 0, 1) v 
3Q(1, 0, 1) v =Q(2, 0, 1) 33. a) Let T(x) be the predicate 
that x can learn new tricks, and let the domain be old dogs. 
Original is 3x T(x). Negation is Vx =T (x): “No old dogs can 
learn new tricks.” b) Let C(x) be the predicate that x knows 
calculus, and let the domain be rabbits. Original is ~3x C(x). 
Negation is 4x C(x): “There is a rabbit that knows calculus.” 
c) Let F(x) be the predicate that x can fly, and let the domain 
be birds. Original is Vx F(x). Negation is 3x ~F (x): “There 
is a bird who cannot fly.” d) Let T(x) be the predicate that 
x can talk, and let the domain be dogs. Original is =3x T(x). 
Negation is 3x T(x): “There is a dog that talks.” e) Let F(x) 
and R(x) be the predicates that x knows French and knows 
Russian, respectively, and let the domain be people in this 
class. Original is 7Ax(F(x) A R(x)). Negation is 4x(F(x) A 
R(x)): “There is someone in this class who knows French and 
Russian.” 35. a) ax(x < 1) b)ax(x > 2) c)Yx(x < 4) 


d) Yxx > 0) e)Ar(x > -1I AG K< 2)) Yxa > 4) 


A (x < 7)) 37.a) Thereisnocounterexample. b)x =0 
c)x=2 39. a) Yx((F(x, 25,000) v S(x, 25)) > E(x)), where 
E(x) is “Person x qualifies as an elite flyer in a given year,” 
F(x, y) is “Person x flies more than y miles in a given year,” 
and S(x, y) is “Person x takes more than y flights in a given 
year” b) Vx((M(x) A T(x, 3)) V M(x) A T(x, 3.5))) > Q(x)), 
where Q(x) is “Person x qualifies for the marathon,” M(x) 
is “Person x is a man,” and T(x, y) is “Person x has run the 
marathon in less than y hours” c) M — ((H(60) v (H(45) A 
T)) A Vy G(B, y)), where M is the proposition “The student 
received a masters degree,” H(x) is “The student took at least 
x course hours,” T is the proposition “The student wrote a 
thesis,” and G(x, y) is “The person got grade x or higher in 


course y” d) 4x ((T(x, 21) A G(x, 4.0)), where T(x, y) is “Per- 
son x took more than y credit hours” and G(x, p) is “Person x 
earned grade point average p” (we assume that we are talking 
about one given semester) 41. a) If there is a printer that 
is both out of service and busy, then some job has been lost. 
b) If every printer is busy, then there is a job in the queue. 
c) If there is a job that is both queued and lost, then some 
printer is out of service. d) If every printer is busy and every 
job is queued, then some job is lost. 43. a) (Ax F(x, 10)) = 
dx S(x), where F(x, y) is “Disk x has more than y kilobytes 
of free space,” and S(x) is “Mail message x can be saved” 
b) (ax A) > Vx(Q(x) > T(x)), where A(x) is “Alert x 
is active,” Q(x) is “Message x is queued,” and T(x) is “Mes- 
sage x is transmitted” c¢) Vx((x # main console) > T(x)), 
where T(x) is “The diagnostic monitor tracks the status of sys- 
tem x” d) Vx(AL(x) > B(x)), where L(x) is “The host of the 
conference call put participant x on a special list” and B(x) 
is “Participant x was billed” 45. They are not equivalent. 
Let P(x) be any propositional function that is sometimes true 
and sometimes false, and let Q(x) be any propositional func- 
tion that is always false. Then Vx(P(x) > Q(x)) is false but 
VxP(x) > VxQ(x) is true. 47. Both statements are true pre- 
cisely when at least one of P(x) and Q(x) is true for at least 
one value of xin the domain. 49. a) If A is true, then both 
sides are logically equivalent to VxP(x). If A is false, the left- 
hand side is clearly false. Furthermore, for every x, P(x) AA 
is false, so the right-hand side is false. Hence, the two sides 
are logically equivalent. b) If A is true, then both sides are 
logically equivalent to 3x P(x). If A is false, the left-hand side 
is clearly false. Furthermore, for every x, P(x) A A is false, 
so Ax(P(x) A A) is false. Hence, the two sides are logically 
equivalent. 51. We can establish these equivalences by ar- 
guing that one side is true if and only if the other side is true. 
a) Suppose that A is true. Then for each x, P(x) > A is true; 
therefore, the left-hand side is always true in this case. By sim- 
ilar reasoning the right-hand side is always true in this case. 
Therefore, the two propositions are logically equivalent when 
A is true. On the other hand, suppose that A is false. There 
are two subcases. If P(x) is false for every x, then P(x) > A 
is vacuously true, so the left-hand side is vacuously true. The 
same reasoning shows that the right-hand side is also true, 
because in this subcase 4xP(x) is false. For the second sub- 
case, suppose that P(x) is true for some x. Then for that x, 
P(x) — A is false, so the left-hand side is false. The right- 
hand side is also false, because in this subcase 4xP(x) is true 
but A is false. Thus, in all cases, the two propositions have 
the same truth value. b) If A is true, then both sides are triv- 
ially true, because the conditional statements have true con- 
clusions. If A is false, then there are two subcases. If P(x) is 
false for some x, then P(x) —> A is vacuously true for that x, 
so the left-hand side is true. The same reasoning shows that 
the right-hand side is true, because in this subcase VxP(x) 
is false. For the second subcase, suppose that P(x) is true 
for every x. Then for every x, P(x) — A is false, so the 
left-hand side is false (there is no x making the conditional 
statement true). The right-hand side is also false, because it 
is a conditional statement with a true hypothesis and a false 


conclusion. Thus, in all cases, the two propositions have 
the same truth value. 53.To show these are not logi- 
cally equivalent, let P(x) be the statement “x is positive,” 
and let Q(x) be the statement “x is negative” with domain 
the set of integers. Then 3x P(x) A Ax Q(x) is true, but 
Ax(P(x) A Q(x)) is false. 55.a) True b) False, unless 
the domain consists of just one element or the hypothe- 
sis is false. c)True 57.a) Yes b)No_ c)juana, kiko 
d) math273, cs301 e)juana, kiko 59. sibling(X,Y) 
:-  mother(M,X), mother(M,Y), father(F,X), 
father(F,Y) 61.a) Vx(P@) > 7Q(x))_ b) Vx(O(x) > 
R(x)) ce) Vx(P(x) > 7AR(x)) d) The conclusion does not follow. 
There may be vain professors, because the premises do not 
rule out the possibility that there are other vain people besides 
ignorant ones. 63. a) Vx(P(x) > 7Q(x)) b) Yx(R(x) > 7S(x)) 
c) Vx(7Q(x) > S(x)) d) Vx(P (x) > AR(x)) e) The conclusion 
follows. Suppose x is a baby. Then, by the first premise, x is 
illogical, so by the third premise, x is despised. The second 
premise says that if x could manage a crocodile, then x would 
not be despised. Therefore, x cannot manage a crocodile. 
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1. a) For every real number x there exists a real number y such 
that x is less than y. b) For every real number x and real num- 
ber y, if x and y are both nonnegative, then their product is non- 
negative. c) For every real number x and real number y, there 
exists areal number z such that xy =z. 3. a) There is some 
student in your class who has sent a message to some student 
in your class. b) There is some student in your class who 
has sent a message to every student in your class. c) Every 
student in your class has sent a message to at least one student 
in your class. d) There is a student in your class who has been 
sent a message by every student in your class. e) Every stu- 
dent in your class has been sent a message from at least one 
student in your class. f) Every student in the class has sent 
a message to every student in the class. 5. a) Sarah Smith 
has visited www.att.com. b) At least one person has visited 
www.imdb.org. c) Jose Orez has visited at least one website. 
d) There is a website that both Ashok Puri and Cindy Yoon 
have visited. e) There is a person besides David Belcher who 
has visited all the websites that David Belcher has visited. 
f) There are two different people who have visited exactly 
the same websites. 7.a) Abdallah Hussein does not like 
Japanese cuisine. b) Some student at your school likes Ko- 
rean cuisine, and everyone at your school likes Mexican cui- 
sine. €) There is some cuisine that either Monique Arsenault 
or Jay Johnson likes. d) For every pair of distinct students at 
your school, there is some cuisine that at least one them does 
not like. e) There are two students at your school who like 
exactly the same set of cuisines. f) For every pair of students 
at your school, there is some cuisine about which they have 
the same opinion (either they both like it or they both do not 
like it). 9. a) VxL(x, Jerry) b) Yx3yL(x, y) ©) dyVxL(y, y) 
d) Vxay7L@, y) e) Ax7L (Lydia, x) f) axYyaLy, x) 
g) Ix(YyLO, x) A Yz((YwL(w, z) >z = x)) b) ąax3y& + 
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yAL(Lynn, x) A L(Lynn, y) A Yz(L(Lynn, z) > (z =xVz=y))) 
i YxLx x) jax Vy (Lx y) x= y) 1l. a)A(Lois, 
Professor Michaels) b) Vx(S(x) — A(x, Professor Gross)) 
c) Yx(F(x) > (A(x, Professor Miller) V A(Professor Miller, 
x))) d) Ix(SŒ) AVY (FO) > TAG, y))) e) Ix(F Œ) A YYSQ) > 
AQ, x) Ð WEFO) mAx(S@) v AG; y))) 8) xF) A 
VFO) A O # x) >Ak, y))) h) ASQ) A WFO) > 
~A(y, x))) 13. a) —M (Chou, Koko) b) ~M(Arlene, Sarah)^ 
aT(Arlene, Sarah) c) ~M (Deborah, Jose) d) VxM(x, Ken) 
e) VxnT(x, Nina) f) Vx(T(x, Avi) V M(x, Avi)) g) AxVy(y # 
x > M(x, y) b)arwyy # x > (MG, y) v Tœ y)) 
i) Axdy(x 4 yA M(x, y) A MO, xX)) j) IxM(x, x) k) IxYy(x 4 
y > COM, y) MTO, x))) D YxEy& # YA(MG, x» VTO, x)))) 
m) 3x3y(x # yA M(x y) a TẸ, xX) n)Axdya # VA 
W2(Z#xAz#yY) > M x z) V M Q, 2) V Tœ z) Y TO, 2) 
15. a) VxP(x), where P(x) is “x needs a course in discrete 
mathematics” and the domain consists of all computer sci- 
ence students b) 4xP(x), where P(x) is “x owns a personal 
computer” and the domain consists of all students in this class 
c) VxayP(%, y), where P(x, y) is “x has taken y,” the domain 
for x consists of all students in this class, and the domain for y 
consists of all computer science classes d) 3x3yP(x, y), where 
P(x, y) and domains are the same as in part (c) e) VxVyP(, y), 
where P(x, y) is “x has been in y,” the domain for x consists 
of all students in this class, and the domain for y consists of 
all buildings on campus f) 3x3yYz(P(z, y) > Q(x, z)), where 
P(z, y) is “zis in y” and Q(x, z) is “x has been in z”; the domain 
for x consists of all students in the class, the domain for y con- 
sists of all buildings on campus, and the domain of z consists 
of all rooms. g) VxVydz(P(z, y) A Q(x z)), with same en- 
vironment as in part (f) 17. a) Vudm(A(u, m) A Yn(n # 
m > A(u, n))), where A(u, m) means that user u has 
access to mailbox m b) 3pVe(H(e) ^A S(p, running)) 
—> S (kernel, working correctly), where H(e) means that 
error condition e is in effect and S(x, y) means that the 
status of x is y c¢)VuYs(E(s, .edu) — A(u, s)), where 
E(s, x) means that website s has extension x, and A(u, s) 
means that user u can access website s d)idxdyx + 
yA Yz((Ys M(z, s)) © (z =x V z = y))), where M(a, b) means 
that system a monitors remote server b 19. a) VxVy((x < 
OAWY < 0) > x+y < 0) b) xy (x>0)A 
>00 > (-y>0)) o) VxVy +y > +4 y)) 
d) YxYy (l| = |x|ly|)) 21. Vxdadbdacdd (x > 0) > 
x =&@ +b? +c? +d’), where the domain consists of all in- 
tegers 23. a) Vx Vy (x < 0) A GY < 0) > (ay > 0) 
b) Vx(x —x = 0) c) Vxdadb(a £ b A Yce(° = x e (C= aV 
c =b) d)Vx((x < 0) > 7A3y(x = y*)) 25. a) There 
is a multiplicative identity for the real numbers. b) The 
product of two negative real numbers is always a positive 
real number. c) There exist real numbers x and y such that 
x? exceeds y but x is less than y. d) The real numbers are 
closed under the operation of addition. 27. a) True b) True 
c) True d) True e) True f) False g) False h) True i) False 
29.a) P(1,1) A P(,2) A P(1,3) A P(2,1) A P,2)A 
PQ, 3) A PB, 1) A PG, 2) A PB, 3) b) Pd, Dv 
Pd, 2) v Pd, 3) v PQ, 1) v PQ, 2) v PQ, 3) v P(3,1) v 
PB, 2) v PG, 3) oe) (Pd, 1) A Pd, 2) A Pd, 3)) v 
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(P(2, 1) A P(2, 2) A P(2, 3)) v (PG, 1) A P3, 2) A PG, 3)) 
d) (Pd, 1) v P2,1) v PB, 1)) A (Pd, 2) v P, 2) Vv 
P(3, 2)) A (PC, 3) v PQ, 3) v P(3, 3)) 31. a) AxVydz aT 
(x y, z) b) AxVy7P@, y) A AxVy 7 QO, y) c) AxVy 
OP, y) V Vz aR, y, z) d) AxVy(P@ y) A Qa, y)) 
33. a) dxdy7P(x, y) b) 3yYxaP(x, y) c) dydx(7P(x, 
y) A 7QGQ, y)) d (WxVyP@ y) Vv axe, y) 
e) Ax(Vydz7P(x, y, z) V Vzdy7P(x%, y, Z)) 35. Any do- 
main with four or more members makes the statement true; 
any domain with three or fewer members makes the state- 
ment false. 37.a) There is someone in this class such 
that for every two different math courses, these are not 
the two and only two math courses this person has taken. 
b) Every person has either visited Libya or has not visited 
a country other than Libya. c) Someone has climbed ev- 
ery mountain in the Himalayas. d) There is someone who 
has neither been in a movie with Kevin Bacon nor has been 
in a movie with someone who has been in a movie with 


Kevin Bacon. 39.a)x = 2 y = -2 b)x = —4 
c)x = 17,y = -1 41. YxYyYz((xy)z = x- 0-2) 
43. Ym Vb (m # 0 > Ax(mx + b = 0 A Yw(mw + b = 
0 > w = XX) 45.a)True b)False c) True 


47. a(axVyP(x, y)) e Yx(YyP(x, y) e  VxdyrP(x,y) 
49. a) Suppose that YxP(x) A IxQ(x) is true. Then P(x) is true 
for all x and there is an element y for which Q(y) is true. Be- 
cause P(x) A Q(y) is true for all x and there is a y for which 
Q(y) is true, Vxdy(P(x) A Q(y)) is true. Conversely, suppose 
that the second proposition is true. Let x be an element in the 
domain. There is a y such that Q(y) is true, so 4xQ(x) is true. 
Because VxP(x) is also true, it follows that the first proposi- 
tion is true. b) Suppose that VxP(x) V 4xQ(x) is true. Then 
either P(x) is true for all x, or there exists a y for which Q(y) 
is true. In the former case, P(x) V Q(y) is true for all x, so 
Vxdy(P(x) V Q(y)) is true. In the latter case, Q(y) is true for a 
particular y, so P(x) V Q) is true for all x and consequently 
Vxdy(P(x) v Q())) is true. Conversely, suppose that the sec- 
ond proposition is true. If P(x) is true for all x, then the first 
proposition is true. If not, P(x) is false for some x, and for this 
x there must be a y such that P(x) V Q(y) is true. Hence, Q(y) 
must be true, so SyQ(y) is true. It follows that the first propo- 
sition must hold. 51. We will show how an expression can 
be put into prenex normal form (PNF) if subexpressions in it 
can be put into PNF. Then, working from the inside out, any 
expression can be put in PNF. (To formalize the argument, 
it is necessary to use the method of structural induction that 
will be discussed in Section 5.3.) By Exercise 49 of Section 
1.3, we can assume that the proposition uses only V and ~ as 
logical connectives. Now note that any proposition with no 
quantifiers is already in PNF. (This is the basis case of the 
argument.) Now suppose that the proposition is of the form 
QxP(x), where Q is a quantifier. Because P(x) is a shorter 
expression than the original proposition, we can put it into 
PNF. Then Qx followed by this PNF is again in PNF and 
is equivalent to the original proposition. Next, suppose that 
the proposition is of the form =P. If P is already in PNF, 
we slide the negation sign past all the quantifiers using the 
equivalences in Table 2 in Section 1.4. Finally, assume that 


proposition is of the form P V Q, where each of P and Q is in 
PNF. If only one of P and Q has quantifiers, then we can use 
Exercise 48 in Section 1.4 to bring the quantifier in front of 
both. If both P and Q have quantifiers, we can use Exercise 
47 in Section 1.4, Exercise 48, or part (b) of Exercise 49 to 
rewrite P V Q with two quantifiers preceding the disjunction 
of a proposition of the form R Vv S, and then put R V S into 
PNF. 


Section 1.6 


1. Modus ponens; valid; the conclusion is true, because 
the hypotheses are true. 3. a) Addition b) Simplification 
c) Modus ponens d) Modus tollens e) Hypothetical syllo- 
gism 5. Let w be “Randy works hard,” let d be “Randy is 
a dull boy,” and let j be “Randy will get the job.” The hy- 
potheses are w, w > d, and d —> ~j. Using modus ponens 
and the first two hypotheses, d follows. Using modus ponens 
and the last hypothesis, =j, which is the desired conclusion, 
“Randy will not get the job,” follows. 7. Universal instan- 
tiation is used to conclude that “If Socrates is a man, then 
Socrates is mortal.” Modus ponens is then used to conclude 
that Socrates is mortal. 9. a) Valid conclusions are “I did 
not take Tuesday off,’ “I took Thursday off,’ and “It rained 
on Thursday.” b) “I did not eat spicy foods and it did not 
thunder” is a valid conclusion. c) “I am clever” is a valid 
conclusion. d) “Ralph is not a CS major” is a valid conclu- 
sion. e) “That you buy lots of stuff is good for the U.S. and 
is good for you” is a valid conclusion. f) “Mice gnaw their 
food” and “Rabbits are not rodents” are valid conclusions. 
11. Suppose that p,, P2, ..., P, are true. We want to establish 
that q > r is true. If q is false, then we are done, vacuously. 
Otherwise, q is true, so by the validity of the given argument 
form (that whenever p}, P2, ..., Pw q are true, then r must be 
true), we know that ris true. 13. a) Let c(x) be “x is in this 
class,” j(x) be “x knows how to write programs in JAVA,” 
and h(x) be “x can get a high-paying job.” The premises are 
c(Doug), j(Doug), Vx(j(x) > h(x)). Using universal instan- 
tiation and the last premise, (Doug) — /(Doug) follows. 
Applying modus ponens to this conclusion and the second 
premise, h(Doug) follows. Using conjunction and the first 
premise, c(Doug) A h(Doug) follows. Finally, using existen- 
tial generalization, the desired conclusion, Ax(c(x) A h(x)) 
follows. b) Let c(x) be “x is in this class,” w(x) be “x enjoys 
whale watching,” and p(x) be “x cares about ocean pollution.” 
The premises are 4x(c(x) A w(x)) and Vx(w(x) > p(x)). From 
the first premise, c(y) A w(y) for a particular person y. Using 
simplification, w(y) follows. Using the second premise and 
universal instantiation, w(y) — p(y) follows. Using modus 
ponens, p(y) follows, and by conjunction, c(y) A p(y) follows. 
Finally, by existential generalization, the desired conclusion, 
Ax(c(x) A p(x)), follows. c) Let c(x) be “x is in this class,” p(x) 
be “x owns a PC,” and w(x) be “x can use a word-processing 
program.” The premises are c(Zeke), Vx(c(x) — p(x)), and 


Vx(p(x) > w(x)). Using the second premise and universal 
instantiation, c(Zeke) — p(Zeke) follows. Using the first 
premise and modus ponens, p(Zeke) follows. Using the third 
premise and universal instantiation, p(Zeke) > w(Zeke) fol- 
lows. Finally, using modus ponens, w(Zeke), the desired con- 
clusion, follows. d) Let j(x) be “x is in New Jersey,” f(x) be “x 
lives within 50 miles of the ocean,” and s(x) be “x has seen the 
ocean.” The premises are Vx(j(x) > f(x)) and Ax(j(x) Aas(x)). 
The second hypothesis and existential instantiation imply 
that j(y) A —s(y) for a particular person y. By simplification, 
Jy) for this person y. Using universal instantiation and the 
first premise, j(y) > f(y), and by modus ponens, f(y) fol- 
lows. By simplification, -s(y) follows from j(y) A 7s(y). So 
fO) A 7s() follows by conjunction. Finally, the desired con- 
clusion, 4x(f(x)A 75(x)), follows by existential generalization. 
15. a) Correct, using universal instantiation and modus po- 
nens b) Invalid; fallacy of affirming the conclusion c) Invalid; 
fallacy of denying the hypothesis d) Correct, using universal 
instantiation and modus tollens 17. We know that some x 
exists that makes H(x) true, but we cannot conclude that Lola 
is one such x. 19. a) Fallacy of affirming the conclusion 
b) Valid argument using modus tollens c) Fallacy of denying 
the hypothesis 21. By the second premise, there is some 
lion that does not drink coffee. Let Leo be such a creature. By 
simplification we know that Leo is a lion. By modus ponens 
we know from the first premise that Leo is fierce. Hence, 
Leo is fierce and does not drink coffee. By the definition of 
the existential quantifier, there exist fierce creatures that do 
not drink coffee, that is, some fierce creatures do not drink 
coffee. 23. The error occurs in step (5), because we cannot 
assume, as is being done here, that the c that makes P true 
is the same as the c that makes Q true. 25. We are given 
the premises Vx(P(x) > Q(x)) and =Q(a). We want to show 
=P(a). Suppose, to the contrary, that =P(a) is not true. Then 
P(a) is true. Therefore, by universal modus ponens, we have 
Q(a). But this contradicts the given premise ~Q(a). There- 
fore, our supposition must have been wrong, and so —P(a) is 
true, as desired. 


27. Step Reason 
1. Vx(P(@x) A R(x)) Premise 
2. P(a) A R(a) Universal instantiation from (1) 


3. P(a) Simplification from (2) 
4. Vx(P(x) > Premise 
(Q) A S(x))) 
5. O(a) A S(a) Universal modus ponens from(3) 
and (4) 
6. S(a) Simplification from (5) 
7. R(a) Simplification from (2) 


8. R(a) A S(a) Conjunction from (7) and (6) 
9. Vx(R(x) A S(x)) Universal generalization from (5) 
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29. Step Reason 
1. Sx4P(x) Premise 
2. aP(c) Existential instantiation from (1) 
3. Vx(P(x) V O(x)) Premise 
4. P(c) V O(c) Universal instantiation from (3) 
5. O(c) Disjunctive syllogism from (4) 
and (2) 
6. Vx(=Q(x) V S(x)) Premise 
7. 7Q(c) V S(c) Universal instantiation from (6) 
8. S(c) Disjunctive syllogism from (5) 
and (7) 
9. Vx(R(x) > =S(x)) Premise 
10. R(c) > =S(c) Universal instantiation from (9) 
11. a=R(c) Modus tollens from (8) and (10) 
12. AxAR(x) Existential generalization from 
(11) 
31. Let p be “It is raining”; let q be “Yvette has her umbrella’; 
let r be “Yvette gets wet.” Assumptions are =p V q, 7g V `r, 
and p V ^r. Resolution on the first two gives =p V ^r. Res- 
olution on this and the third assumption gives ^r, as desired. 
33. Assume that this proposition is satisfiable. Using resolu- 
tion on the first two clauses enables us to conclude q V q; in 
other words, we know that q has to be true. Using resolution 
on the last two clauses enables us to conclude =q V ~q; in 
other words, we know that ~q has to be true. This is a contra- 
diction. So this proposition is not satisfiable. 35. Valid 


Section 1.7 


l. Let n = 2k + 1 adm = 21 + 1 be odd inte- 
gers. Then n+m=2(k+1+1) is even. 3. Suppose that n 
is even. Then n = 2k for some integer k. Therefore, r= 


(2k) =4k? = 2(2k?). Because we have written n? as 2 times an 
integer, we conclude that n? is even. 5. Direct proof: Sup- 
pose that m +n and n + p are even. Then m+n = 2s for some 
integer s and n +p = 2t for some integer t. If we add these, we 
get m+p+2n = 2s + 2t. Subtracting 2n from both sides and 
factoring, we have m + p = 2s + 2t — 2n = 2(s+t—n). 
Because we have written m + p as 2 times an integer, we 
conclude that m + p is even. 7. Because n is odd, we can 
write n = 2k + 1 for some integer k. Then (k + 1)? — k? = 
K +2k+1—k = 2k+1 =n. 9. Suppose that r is 
rational and / is irrational and s = r + i is rational. Then by 
Example 8, s + (—r) = i is rational, which is a contradiction. 
11. Because V2 . V2 = 2 is rational and V2 is irrational, the 
product of two irrational numbers is not necessarily irrational. 
13. Proof by contraposition: If 1/x were rational, then by def- 
inition 1/x = p/q for some integers p and q with q # 0. 
Because 1/x cannot be 0 (if it were, then we’d have the con- 
tradiction 1 = x - 0 by multiplying both sides by x), we know 
that p # 0. Now x = 1/(1/x) = 1/(p/q) = q/p by the 
usual rules of algebra and arithmetic. Hence, x can be written 
as the quotient of two integers with the denominator nonzero. 
Thus, by definition, x is rational. 15. Assume that yx were 
rational. Then, because the product of two rational numbers 
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is rational, (Va = xis also rational. This contradicts the 
hypothesis that x is irrational. 17. Assume that it is not true 
thax > 1 ory > 1. Thenx < landy < 1. Adding these 
two inequalities, we obtain x + y < 2, which is the negation 
ofx+y>2. 19. a) Assume that n is odd, so n = 2k + 1 for 
some integer k. Then n? + 5 = 2(4k? + 6k? + 3k + 3). Because 
n? + 5 is two times some integer, it is even. b) Suppose that 
n? + 5 is odd and n is odd. Because n is odd and the prod- 
uct of two odd numbers is odd, it follows that n? is odd and 
then that n? is odd. But then 5 = (n? + 5) — n? would have 
to be even because it is the difference of two odd numbers. 
Therefore, the supposition that n? + 5 and n were both odd 
is wrong. 21. The proposition is vacuously true because 0 
is not a positive integer. Vacuous proof. 23. P(1) is true be- 
cause (a+b)! = a+b > a! +b! = a+b. Direct proof. 25. If 
we chose 9 or fewer days on each day of the week, this would 
account for at most 9-7 = 63 days. But we chose 64 days. This 
contradiction shows that at least 10 of the days we chose must 
be on the same day of the week. 27. Suppose by way of con- 
tradiction that a/b is a rational root, where a and b are integers 
and this fraction is in lowest terms (that is, a and b have no 
common divisor greater than 1). Plug this proposed root into 
the equation to obtain a? /b? + a/b + 1 = 0. Multiply through 
by D? to obtain a? +ab? +b? = 0. If a and b are both odd, then 
the left-hand side is the sum of three odd numbers and there- 
fore must be odd. If a is odd and b is even, then the left-hand 
side is odd +even +even, which is again odd. Similarly, if a is 
even and b is odd, then the left-hand side is even + even +odd, 
which is again odd. Because the fraction a/b is in simplest 
terms, it cannot happen that both a and b are even. Thus, in all 
cases, the left-hand side is odd, and therefore cannot equal 0. 
This contradiction shows that no such root exists. 29. First, 
assume that n is odd, so that n = 2k + 1 for some integer k. 
Then 5n +6 = 5(2k+1)+6 = 10k+ 11 = 2(5k+5)+1. Hence, 
5n + 6 is odd. To prove the converse, suppose that n is even, 
so that n = 2k for some integer k. Then 5n + 6 = 10k +6 = 
2(5k + 3), so 5n + 6 is even. Hence, n is odd if and only if 
5n+6isodd. 31. This proposition is true. Suppose that m is 
neither 1 nor —1. Then mn has a factor m larger than 1. On the 
other hand, mn = 1, and 1 has no such factor. Hence, m = 1 or 
m = —1. Inthe first case n = 1, and in the second case n = —1, 
because n = 1/m. 33. We prove that all these are equivalent 
to x being even. If x is even, then x = 2k for some integer k. 
Therefore, 3x+2 = 3-2k+2 = 6k+2 = 2(3k+1), which is even, 
because it has been written in the form 2t, where t = 3k + 1. 
Similarly, x+5 = 2k+5 = 2k+4+1 =2(k+2)+1,sox+5 is 
odd; and x? = (2k)? = 2(2k7), so x? is even. For the converses, 
we will use a proof by contraposition. So assume that x is not 
even; thus, x is odd and we can write x = 2k+ 1 for some inte- 
ger k. Then 3x+2 = 3(2k+1)+2 = 6k+5 = 2(3k+2)+1, which 
is odd (i.e., not even), because it has been written in the form 
2t+1, where t = 3k+2. Similarly, x+5 = 2k+1+5 = 2(k+3), 
so x + 5 is even (i.e., not odd). That x” is odd was already 
proved in Example 1. 35. We give proofs by contraposition 
of (i) > (ii), (ii) > (i), (i) > (iii), and (iii) > (i). For the 
first of these, suppose that 3x + 2 is rational, namely, equal to 


p/q for some integers p and q with q # 0. Then we can write 
x = ((p/q) —2)/3 = (p —2q)/(q), where 3g # 0. This shows 
that x is rational. For the second conditional statement, sup- 
pose that x is rational, namely, equal to p/q for some integers 
p and q with q + 0. Then we can write 3x + 2 = (3p + 2q)/q, 
where q + 0. This shows that 3x + 2 is rational. For the third 
conditional statement, suppose that x/2 is rational, namely, 
equal to p/q for some integers p and q with q # 0. Then 
we can write x = 2p/q, where g # 0. This shows that 
x is rational. And for the fourth conditional statement, sup- 
pose that x is rational, namely, equal to p/q for some integers 
p and q with gq # 0. Then we can write x/2 = p/(2q), 
where 2q # 0. This shows that x/2 is rational. 37. No 
39. Suppose that p) > Py > P2 > Ps > P3 > Pi 
To prove that one of these propositions implies any of the 
others, just use hypothetical syllogism repeatedly. 41. We 
will give a proof by contradiction. Suppose that a4, a», ..., A, 
are all less than A, where A is the average of these numbers. 
Then a, + a, + = +a, < nA. Dividing both sides by n shows 
that A = (a, +a, + +++ +a,)/n < A, which is a contradiction. 
43. We will show that the four statements are equivalent by 
showing that (i) implies (ii), (di) implies (iii), (iii) implies 
(iv), and (iv) implies (i). First, assume that n is even. Then 
n = 2k for some integer k. Then n + 1 = 2k + 1, so 
n + 1 is odd. This shows that (i) implies (ii). Next, suppose 
that n + 1 is odd, son + 1 = 2k + 1 for some integer k. 
Then 3n + 1 = 2n + (n+ 1) = 2(n + k) + 1, which 
shows that 3n + 1 is odd, showing that (ii ) implies (iii). Next, 
suppose that 3n + 1 is odd, so 3n + 1 = 2k + 1 for some 
integer k. Then 3n = (2k + 1)— 1 = 2k, so 3n is even. 
This shows that (iii) implies (iv). Finally, suppose that n is 
not even. Then n is odd, son = 2k + 1 for some integer k. 
Then 3n = 3(2k + 1) = 6k + 3 = 2(3k + 1) + 1, so 3n is odd. 
This completes a proof by contraposition that (iv) implies (i). 


Section 1.8 


1.1? +1=2 > 2=2!;2 +1 = 5 > 4=2;3+1= 
10>8=23;4+1=17>16=2+ 3. We must show 
that for all positive integers x it is not true that x? = 100. 
Case (i): If x < 4, then x? < 64, sox? # 100. Case (ii): 
Ifx > 5, thenx® > 125,soxr # 100. 5. Ifx < y, then 
max(x, y) + mina, y) = y+x = x+y.Ifx > y, then 
max(x, y) + min(x, y) = x + y. Because these are the only two 
cases, the equality always holds. 7. Because |x—y| = |y—x|, 
the values of x and y are interchangeable. Therefore, with- 
out loss of generality, we can assume that x > y. Then 
(x+y—(x—y))/2 =(xt+y—x+y)/2 = 2y/2 = y = min y). 
Similarly, (n+ y+(x—y))/2 = x+y +x—y)/2 = 2x/2 = x = 
max(x, y). 9. There are four cases. Case 1: x > 0 and y > 0. 
Then |x| + |y] = x+y = |x + y|. Case 2: x < 0 and y < 0. 
Then |x| + |y] = —x + (-y) = —(x + y) = |x + y| because 
x+y <0. Case 3: x > 0 and y < 0. Then |x| + |y| =x+(—y). 
If x > —y, then |x + y| = x + y. But because y < 0, —y > y, 
so |x| + |y| = x+(-y) > x+y = [x+y]. If x < —y, then 
|x + y| = —(x + y) = —x + (—y). But because x > 0, x > —x, 


so |x| + [y| =x+(-y) > —x+ (—y) = |x + y|. Case 4: x < 0 
and y > 0. Identical to Case 3 with the roles of x and y re- 
versed. 11. 10,001, 10,002,..., 10,100 are all nonsquares, 
because 1002 = 10,000 and 1012 = 10,201; constructive. 
13.8 = 2 and9 = 32 15.Letx = 2andy = v2. If 
v= 272 is irrational, we are done. If not, then let x = 2v2 and 
y = 2/4. Then x” = (2V2)V2/4 = 2V2V2/4 = 21⁄2 = 4/7, 
17. a) This statement asserts the existence of x with a certain 
property. If we let y = x, then we see that P(x) is true. If y 
is anything other than x, then P(x) is not true. Thus, x is the 
unique element that makes P true. b) The first clause here says 
that there is an element that makes P true. The second clause 
says that whenever two elements both make P true, they are 
in fact the same element. Together these say that P is satisfied 
by exactly one element. c) This statement asserts the exis- 
tence of an x that makes P true and has the further property 
that whenever we find an element that makes P true, that ele- 
ment is x. In other words, x is the unique element that makes 
P true. 19. The equation |a—c| = |b—c| is equivalent to the 
disjunction of two equations: a — c = b— cora —c = —b +c. 
The first of these is equivalent to a = b, which contradicts the 
assumptions made in this problem, so the original equation is 
equivalent to a — c = —b + c. By adding b + c to both sides 
and dividing by 2, we see that this equation is equivalent to 
c = (a + b)/2. Thus, there is a unique solution. Furthermore, 
this c is an integer, because the sum of the odd integers a and b 
iseven. 21. We are being asked to solve n = (k—2)+(k+3) 
for k. Using the usual, reversible, rules of algebra, we see that 
this equation is equivalent to k = (n—1)/2. In other words, this 
is the one and only value of k that makes our equation true. 
Because n is odd, n — 1 is even, so k is an integer. 23. If xis 
itself an integer, then we can take n = x and e = 0. No other so- 
lution is possible in this case, because if the integer n is greater 
than x, then n is at least x + 1, which would make e > 1. If x 
is not an integer, then round it up to the next integer, and call 
that integer n. Let e = n—x. Clearly 0 < e < 1; this is the only 
e that will work with this n, and n cannot be any larger, be- 
cause € is constrained to be less than 1. 25. The harmonic 
mean of distinct positive real numbers x and y is always less 
than their geometric mean. To prove 2xy/(x + y) < xy, 
multiply both sides by (x + y)/ (24/37) to obtain the equiv- 
alent inequality Vy < (x + y)/2, which is proved in Ex- 
ample 14. 27. The parity (oddness or evenness) of the sum 
of the numbers written on the board never changes, because 
j+ kand |j — k| have the same parity (and at each step we 
reduce the sum by j + k but increase it by |j — k|). There- 
fore the integer at the end of the process must have the same 
parity as 1 + 2 + ++ + (Qn) = n(2n + 1), which is odd 
because nis odd. 29. Without loss of generality we can as- 
sume that n is nonnegative, because the fourth power of an 
integer and the fourth power of its negative are the same. We 
divide an arbitrary positive integer n by 10, obtaining a quo- 
tient k and remainder l, whence n = 10k + J, and / is an 
integer between 0 and 9, inclusive. Then we compute n* in 
each of these 10 cases. We get the following values, where X 
is some integer that is a multiple of 10, whose exact value we 
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do not care about. (10k + 0)* = 10,000k* = 10,000k* + 0, 
(10k + 1)* = 10,000kt 4+ X¥ -B4+X-PR4+X-k 41, 
(10k + 2)* = 10,000 +X -k+X-k?4+X-k + 16, 
(10k + 3)* = 10,000k4 +X -RP4X-hM4+X-k4 81, 
(10k + 4)* = 10,0004 +X -K +X- k +X- k+256, 
(10k + 5)* = 10,000 + X -K +X -kK +X -k+ 625, 
(10k + 6)* = 10,000k4 +X-kK +X-kK +X -k+ 1296, 
(10k + 7} = 10,000 +X- 4+ X-k +X- k+ 2401, 
(10k + 8) = 10,000k4 + X-K +X-K +X- k + 4096, 
(10k + 9) = 10,000k4 +X - kK? +X- k? +X- k+ 6561. 
Because each coefficient indicated by X is a multiple of 10, 
the corresponding term has no effect on the ones digit of the 
answer. Therefore, the ones digits are 0, 1, 6, 1, 6, 5, 6, 1, 6, 
1, respectively, so it is always a 0, 1, 5, or 6. 31. Because 
n? > 100 for all n > 4, we need only note that n = 1, n = 2, 
n = 3, and n = 4 do not satisfy n? +n? = 100. 33. Because 
54 = 625, both x and y must be less than 5. Then xt + 
yt <444+44=512 < 625. 35. If it is not true that a < Wn, 
b < ẹ/n, orc < vn, then a > Vn, b > vn, and c > Vn. 
Multiplying these inequalities of positive numbers together 
we obtain abc < (ny = n, which implies the negation 
of our hypothesis that n = abc. 37. By finding a common 
denominator, we can assume that the given rational numbers 
are a/b and c/b, where b is a positive integer and a and c are 
integers with a < c. In particular, (a + 1)/b < c/b. Thus, 
x=(a+ 14/2) /b is between the two given rational numbers, 


because 0 < V2 < 2. Furthermore, x is irrational, because if 
x were rational, then 2(bx— a) = V2 would be as well, in vio- 
lation of Example 11 in Section 1.7. 39. a) Without loss of 
generality, we can assume that the x sequence is already sorted 
into nondecreasing order, because we can relabel the indices. 
There are only a finite number of possible orderings for the y 
sequence, so if we can show that we can increase the sum (or 
at least keep it the same) whenever we find y; and y; that are 
out of order (i.e., i < j but y, > y;) by switching them, then we 
will have shown that the sum is largest when the y sequence is 
in nondecreasing order. Indeed, if we perform the swap, then 
we have added x;y; +x;y; to the sum and subtracted x;y; + x;y;. 
The net effect is to have added x;y; + x;y; — xj; — xy; = 
(x; — x); — y;), which is nonnegative by our ordering as- 
sumptions. b) Similar to part (a) 41.a)6 > 3 > 10 > 
5 > 16 > 8 > 4 > 2 > 1b)7 > 22 > 11 > 34 > 
17 => 52 > 26 => 13 — 40 > 20 => 10 = 5- 16 > 
8> 4->2>1 017 > 52 - 26 > 13 > 40 > 
20-10 > 5 > 16 > 8 > 4 > 2 > 1 
d) 21 —> 64 > 32 => 16 > 8 ~ 452-1 43. Without 
loss of generality, assume that the upper left and upper right 
corners of the board are removed. Place three dominoes hor- 
izontally to fill the remaining portion of the first row, and fill 
each of the other seven rows with four horizontal dominoes. 
45. Because there is an even number of squares in all, either 
there is an even number of squares in each row or there is an 
even number of squares in each column. In the former case, 
tile the board in the obvious way by placing the dominoes hor- 
izontally, and in the latter case, tile the board in the obvious 
way by placing the dominoes vertically. 47. We can rotate 
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the board if necessary to make the removed squares be 1 and 
16. Square 2 must be covered by a domino. If that domino is 
placed to cover squares 2 and 6, then the following domino 
placements are forced in succession: 5-9, 13-14, and 10-11, 
at which point there is no way to cover square 15. Otherwise, 
square 2 must be covered by a domino placed at 2-3. Then 
the following domino placements are forced: 4-8, 11-12, 6-7, 
5-9, and 10-14, and again there is no way to cover square 15. 
49. Remove the two black squares adjacent to a white corner, 
and remove two white squares other than that corner. Then no 
domino can cover that white corner. 


51. a) 


a) (3) (4) (5) 
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b) The picture shows tilings for the first four patterns. 
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To show that pattern 5 cannot tile the checkerboard, label the 
squares from 1 to 64, one row at a time from the top, from left 
to right in each row. Thus, square 1 is the upper left corner, 
and square 64 is the lower right. Suppose we did have a tiling. 
By symmetry and without loss of generality, we may suppose 
that the tile is positioned in the upper left corner, covering 
squares 1, 2, 10, and 11. This forces a tile to be adjacent to 
it on the right, covering squares 3, 4, 12, and 13. Continue in 
this manner and we are forced to have a tile covering squares 
6, 7, 15, and 16. This makes it impossible to cover square 8. 
Thus, no tiling is possible. 


Supplementary Exercises 


laq->p bqdAp o)-gV-7p d)q e p 3.a)The 
proposition cannot be false unless =p is false, so p is true. If 
p is true and q is true, then -q A (p —> q) is false, so the 
conditional statement is true. If p is true and q is false, then 
p > qis false, so =q A (p > q) is false and the conditional 
statement is true. b) The proposition cannot be false unless 
q is false. If q is false and p is true, then (p V q) A >p is false, 
and the conditional statement is true. If q is false and p is 


false, then (p V q) A >p is false, and the conditional statement 
is true. 5.7g > "pp > q; p > q 7.(.pAqar 
A7AS)\V(PAGA7KAS)V (PATAGATAS)V(APAGATrAS) 
9. Translating these statements into symbols, using the obvi- 
ous letters, we have =at > ng, ag > ~q, r > q, and =t Ar. 
Assume the statements are consistent. The fourth statement 
tells us that =ż must be true. Therefore, by modus ponens with 
the first statement, we know that ~g is true, hence (from the 
second statement), that =q is true. Also, the fourth statement 
tells us that r must be true, and so again modus ponens (third 
statement) makes q true. This is a contradiction: q A7q. Thus, 
the statements are inconsistent. 11. Reject-accept-reject- 
accept, accept-accept-accept-accept, accept-accept-reject- 
accept, reject-reject-reject-reject, reject-reject-accept-reject, 
and reject-accept-accept-accept 13. Aaron is a knave and 
Crystal is a knight; it cannot be determined what Bohan is. 
15. Brenda 17. The premises cannot both be true, because 
they are contradictory. Therefore, it is (vacuously) true that 
whenever all the premises are true, the conclusion is also 
true, which by definition makes this a valid argument. Be- 
cause the premises are not both true, we cannot conclude 
that the conclusion is true. 19. Use the same propositions 
as were given in Section 1.3 for a 9 X 9 Sudoku puzzle, 
with the variables indexed from 1 to 16, instead of from 1 
to 9, and with a similar change for the propositions for the 
4 x 4 blocks: \_) Avy Apai Vier Vier PGT + i 45 + j, n). 
21.a)F bT oc)F dT e)F fT = 23. Many an- 
swers are possible. One example is United States sena- 
tors. 25. Vxdydz(y # z A Yw(P(w, x) e (Ww = yVw = 2))) 
27. a) 7AxP(x) b) ax(P@) A WPO > y = y) 
o) Ax, Ix (Px) A PO) Ax, #X A Yy (PO) > O = x, Vy = 
X))) d) Ax, 3x 3x (P AP) AP(X3)AX, FX AX, F 
X3 AX, $ %3AVYPO) > Y =X VY =H VY = X3))) 
29. Suppose that 4x(P(x) > Q(x)) is true. Then either Q(x) 
is true for some Xp, in which case VxP(x) > Ax Q(x) is true; 
or P(X) is false for some xp, in which case VxP(x) > AxQ(x) 
is true. Conversely, suppose that 4x(P(x) > Q(x)) is false. 
That means that Vx(P(x) A 7=Q(x)) is true, which implies 
VxP(x) and Vx(-Q(x)). This latter proposition is equiva- 
lent to 7=AxQ(x). Thus, VxP(x) > 3xQ(x) is false. 31. No 
33. Vx Vz dy T(x, y, z), where T(x, y, z) is the statement 
that student x has taken class y in department z, where 
the domains are the set of students in the class, the set of 
courses at this university, and the set of departments in the 
school of mathematical sciences 35. d!xd!y T(x, y) and 
AxVz((AyVw(T(z, w) e w = y)) e z = x), where T(x, y) 
means that student x has taken class y and the domain is all 
students in this class 37. P(a) > Q(a) and Q(a) > R(a) 
by universal instantiation; then ~Q(a) by modus tollens and 
=P(a) by modus tollens 39. This is not true. Let x = 2!/3. 
Then x? = 4!/3 is irrational (the proof of this is very similar 
to the proof in Example 11 in Section 1.7), but x = 2 is 
rational. 41. We can give a constructive proof by letting 
m = 10° + 1. Then m? = (10° + 1)? > (10°)? = 101900, 
43. 23 cannot be written as the sum of eight cubes. 45. 223 
cannot be written as the sum of 36 fifth powers. 


CHAPTER 2 


Section 2.1 


1. a) {-1,1} b) {1,2,3,4,5,6,7,8,9,10,11} œ) {0,1,4, 9, 16, 
25, 36, 49, 64,81} d)@ 3. a) [0,5), [0,5] b) (0,5), (0, 5], 
[0, 5), [0, 5] ©) (0, 5), (0, 5], [0, 5), [0, 5], (1, 4], [2, 3] d) (0, 5), 
(0, 5], [0, 5), [0, 5], (1, 41, [2, 3] e) (0, 5), (0, 5], [0, 5), [0, 5], 
(1, 4] £ (0, 5], [0,5] 5. a) The second is a subset of the 
first, but the first is not a subset of the second. b) Neither 
is a subset of the other. c) The first is a subset of the sec- 
ond, but the second is not a subset of the first. 7.a) Yes 
b) No c) No 9.a) Yes b) No c) Yes d) No e) No f) No 
11. a) False b) False c) False d) True e) False f) False 
g) True 13.a) True b)True c) False d) True e) True 
f) False 

15. 


Januar 
ui February 


March 


April 


September October 


November 
December 


17. The dots in certain regions indicate that those regions are 
not empty. 


19. Suppose that x€A. Because ACB, this implies 
that x E€ B. Because B C C, we see that x € C. Because 
x E A implies that x € C, it follows that ACC. 21.a)1 
b) 1 ©¢)2 d)3 23.a) {@ {a}} b) {@ {a}, {b}, {a b}} 
c) {Ø, {Ø}, ({B}}, 10, {O}}} 25.a)8 b) 16 ¢)2 27. For 
the “if? part, given A GC B, we want to show that 
P(A) C P(B), ie., if C C AthenC C B. But this fol- 
lows directly from Exercise 19. For the “only if” part, given 
that P(A) C P(B), we want to show that A C B. Suppose 
a E A. Then {a} CA, so {a} € P(A). Since P(A) C P(B), it 
follows that {a} € P(B), which means that {a} C B. But this 
implies a € B, as desired. 29. a) {(a, y), (b, y), (c, y), (d, y), 
(a, z), (b, z), (c, 2), (d, 23)} b) {0, a), ©, b), ©, ©, O, d), @ a), 
(z b), (z c), (z d)} 31. The set of triples (a, b, c), where a 
is an airline and b and c are cities. A useful subset of this set 
is the set of triples (a, b, c) for which a flies between b and c. 
33.Øø x A = {(x y) |x E€ andy E A} = Ø = {x y) | 
x E Aandy € Ø} = AXØ 35.a){(0, 0), (0, 1), (0, 3), 
(1,0), (1, 1), (1, 3), G, 0), G; 1), G, 3)} b) {1 1), (G, 2), (1, a), 
(1, b), 2, 1), (2, 2), (2, a), (2, b), (a, 1), (a, 2), (a, a), (a, b), 
(b, 1), (b, 2), (b,a), (b, b)} 37.mn 39. m” 41. The ele- 
ments of A X B x C consist of 3-tuples (a, b, c), where a € A, 
b € B, and c E C, whereas the elements of (A X B) x C look 
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like ((a, b), c)—ordered pairs, the first coordinate of which 
is again an ordered pair. 43. This is not true. The simplest 
counterexample is to let A = B = Ø. ThenAxB =Ø 
and P(A x B) = {Ø}, whereas P(A) = P(B) = {Ø} and 
P(A) x P(B) = {(@, Ø)}. Thus, P(A x B) # P(A) x P(B). 
45. a) The square of a real number is never —1. True b) There 
exists an integer whose square is 2. False c) The square of ev- 
ery integer is positive. False d) There is a real number equal 
to its own square. True 47. a) {-1,0,1} b)Z- {0,1} © Ø 
49. We must show that {{a}, {a,b}} = {{c}, {c, d}} if and 
only if a = c and b = d. The “if” part is immediate. So as- 
sume these two sets are equal. First, consider the case when 
a + b. Then { {a}, {a, b}} contains exactly two elements, one 
of which contains one element. Thus, {{c}, {c, d}} must have 
the same property, so c # d and {c} is the element containing 
exactly one element. Hence, {a} = {c}, which implies that 
a = c. Also, the two-element sets {a, b} and {c, d} must be 
equal. Because a = c anda ¢ J, it follows that b = d. Second, 
suppose thata = b. Then { {a}, {a, b}} = {{a}}, a set with one 
element. Hence, {{c}, {c, d}} has only one element, which 
can happen only when c = d, and the set is {{c}}. It then 
follows that a = c and b = d. 51. Let S = {aj,d,...,a,}. 
Represent each subset of S$ with a bit string of length n, where 
the ith bit is 1 if and only if a; € S. To generate all subsets of 
S, list all 2” bit strings of length n (for instance, in increasing 
order), and write down the corresponding subsets. 


Section 2.2 


1. a) The set of students who live within one mile of school 
and walk to classes b) The set of students who live within 
one mile of school or walk to classes (or do both) c) The 
set of students who live within one mile of school but 
do not walk to classes d) The set of students who walk 
to classes but live more than one mile away from school 
3. a) {0,1,2,3,4,5,6} b) {3} c) {1, 2,4,5} d) {0,6} 5.A = 
{x | =@ EA) = {x € AD} ={x | x € A} =A 
TaQAUU=Ha{x|xEAVxEU}={x|xEAVTH= 
{x | T}=Ubd)ANB={x|x CEAAX EDP = {x|xe€ 
AAF}={x|F}=@ 9.a)AUA={x|xeEAVxGA} =U 
DANA = {x]x € AAx éA} = GO ILaAu 
B = {x|xE€AVxE Bh ={x|xEBvxEA}=BUA 
DANB= {x|xEAAxEBS={x|xEBAx EAS = 
BAA 13.Supposex € AN (A U B). Thenx € A and 
x € AUB by the definition of intersection. Because x € A, 
we have proved that the left-hand side is a subset of the right- 
hand side. Conversely, let x € A. Then by the definition of 
union, x E A U B as well. Therefore, x E€ A N (A U B) 
by the definition of intersection, so the right-hand side is 
a subset of the left-hand side. 15.a)x € AUB 
x GAUBZ=7AXKEAVXEB)=ZE7H EA) AAW EB) 
x€AAXEB=EXECAAXEBEXEANB 
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17. Suppose A C B. We must show that every element x of U 
is an element of A U B. Hither x € Aorx €A, andifx €A 
then x € B. Thus, x € A UB in all cases. Conversely, suppose 
that AUB = U, and let x € A. Then x ¢ A, so it must be 
that x € B. This shows that A C B, and the proof is complete. 
19. ayxEANBNC = xgANBAC=x¢éAvx é 
BVx€C=xXEAVXEBVXE C=xE€AUBUC 


m6 [anane| same aa] [ue 
1 11 1 0 0 
1 1 0 0 1 
101 0 1 
1 0 O 0 1 
0 1 1 (0) 1 
010 (0) 1 
0 0 1 (0) 1 
000 (0) 1 


21. a) Both sides equal {x |x E AAxéB}. bbþA=ANU= 
AN(BUB) =(ANB)U(ANB) 23.xE€AU(BUO= 
(x E€ AV(XE (BUC) ZF MEAVAEBVXE 
CO = WEAVXE BVA EO=zx EC AUBUC 
25.xEAU(BNC)=HEAVOEBNO)= 
(xE€A)VWEBAXEQO FEF wCEAVXE 
BAWEAVXEQO=Zzx E€E AUB NAUC 
27. a) {4,6} b) {0,1,2,3,4,5,6,7,8,9,10} œ) {4, 5, 6, 8, 10} 
d) {0,2,4,5,6,7,8,9,10} 29. a) The double-shaded portion 
is the desired set. 


A B 
Z 


C 


b) The desired set is the entire shaded portion. 
A B 


C 


c) The desired set is the entire shaded portion. 
A B 


E 


3l.a)B CA b)A CB C)ANB = Ø d) Nothing, because 
this is always true e) A = B 33.A C B = VWx(x EA > 
x EB) = Wa éB>xgéA = VX EBOxE 

A) = BC A 35. By De Morgan’s law, the left-hand side 
equals (A NBYA(Ba Onan ©). By the commuta- 
tive, associative, and idempotent laws, this simplifies to the 


right-hand side. 37. a) Let (x, y) € A x (B — C), which 
means that x € A and y is an element of B but not C. Thus, 
(x,y) E A x Band (x, y) É A X C, so by the definition 
of set difference, (x, y) € (A xX B) — (A x C). Conversely, 
let (x, y) € (A X B) — (A x C). Then (x, y) E A X B and 
(x, y) A x C. Thus, x € A and y € B, and because x € A, 
it must be that y C. This implies that y € B — C, so in- 
deed (x, y) E A x (B — C). b) Note that the complement in 
the right-hand side must mean with respect to U x U. This is 
not true. For example, let U = {a,b}, A = {a}, B = {b}, 
and C = Ø. Then the left-hand side is {(b, a)}, whereas the 
right-hand side is {(a, a), (b, a), (b, b)}. 39. The set of stu- 
dents who are computer science majors but not mathematics 
majors or who are mathematics majors but not computer sci- 
ence majors 41. An element is in (A U B) — (A N B) if it is 
in the union of A and B but not in the intersection of A and B, 
which means that it is in either A or B but not in both A and 
B. This is exactly what it means for an element to belong to 
A@B. 43.a) A®A = (A-A)U(A-A) =GUG=GD 
b)A® = (A-H)U@—-A) = AUB =A CQAGUH= 
(A—U)U(U—A) = ®VUA=A dA@A = (A-A)U 
(A-A) = AUA=U 45.B=Q 47.Yes 49. Yes 
51. If AUB were finite, then it would have n elements for some 
natural number n. But A already has more than n elements, be- 
cause it is infinite, and A U B has all the elements that A has, 
so A UB has more than n elements. This contradiction shows 
that A U B must be infinite. 53.a) {1,2,3,...,} b) {1} 
55.a)A, b) {0,1} 57.a)Z, {—1, 0,1} b)Z — {0}, Ø 
c) R, [-1 1] d) [l, œ), Ø 59.a) {1, 2, 3, 4, 7, 8, 9, 10} 
b) {2, 4,5, 6,7} c) {1,10} 61. The bit in the ith position of 
the bit string of the difference of two sets is | if the ith bit of 
the first string is 1 and the ith bit of the second string is 0, 
and is 0 otherwise. 63.a) 11 1110 0000 0000 0000 0000 
0000 v 01 1100 1000 0000 0100 0101 0000 = 11 1110 1000 
0000 0100 0101 0000, representing {a, b, c, d, e, g, p, t, v} 
b) 11 1110 0000 0000 0000 0000 0000 A 01 1100 1000 0000 
0100 0101 0000 = 01 1100 0000 0000 0000 0000 0000, rep- 
resenting {b, c,d} ce) (11 1110 0000 0000 0000 0000 0000 v 
00 0110 0110 0001 1000 0110 0110) A (01 1100 1000 0000 
0100 0101 0000 v 00 1010 0010 0000 1000 0010 0111) = 
11 1110 0110 0001 1000 0110 0110 A O1 1110 1010 0000 
1100 0111 0111 =01 11100010 0000 1000 0110 0110, rep- 
resenting {b, c, d, e, i, o, t, u, x, y} d) 11 11100000 0000 0000 
0000 0000 v 01 1100 1000 0000 0100 0101 0000 v 00 1010 
0010 0000 1000 0010 0111 v 00 01100110 0001 1000 0110 
0110 = 11 1110 1110 0001 1100 0111 0111, representing 
{a,b,c,d,e,g,h,i,n,o,p,t,u,v,x%y,z} 65.a) {1, 2, 3, {1, 2, 3}} 
b) {Ø} c) {Ø {Ø}} d) {%, {Ø}, (0, {Ø}}} 67.a) {3-a, 3- 
b, 1-c, 4-d} b) {2-a, 2-b} c) {1-a, 1-c}d){1-b, 4-d} 
e) {5-a, 5-b, 1-c,4-d} 69. These all follow from the defi- 
nitions of the multiset operations. a) False, because the union 
is actually {a, a, a} b) False, because the union is actually 
{a,a,a} c) True d) False, because the correct intersection is 
as stated in part (c) e) True 71. a)0,1 b)1/3,2/3 ¢) 1,0 
d) 1/6,5/6 73. F = {0.4 Alice, 0.1 Brian, 0.6 Fred, 0.9 Os- 
car, 0.5 Rita}, R = {0.6 Alice, 0.2 Brian, 0.8 Fred, 0.1 Oscar, 


0.3 Rita} 75. {0.4 Alice, 0.8 Brian, 0.2 Fred, 0.1 Oscar, 
0.5 Rita} 


Section 2.3 


1.a) f(0) is not defined. b) f(x) is not defined for x < 0. c) f(x) 
is not well defined because there are two distinct values as- 
signed to each x. 3. a) Nota function b) A function c) Nota 
function 5. a) Domain the set of bit strings; range the set of 
integers b) Domain the set of bit strings; range the set of even 
nonnegative integers c) Domain the set of bit strings; range 
the set of nonnegative integers not exceeding 7 d) Domain 
the set of positive integers; range the set of squares of pos- 
itive integers = {1, 4, 9, 16, ...} | 7.a) Domain Z*xZ*; 
range Z* b) Domain Z*; range {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} 
c) Domain the set of bit strings; range N d) Domain the set of 
bit strings; rangeN 9.a) 1 b)0c)0 d)—1 e)3 f)-1 g)2 
h)1 11. Only the function in part (a) 13. Only the func- 
tions in parts (a) and (d) 15. a) Onto b) Not onto c) Onto 
d) Not onto e) Onto 17. a) Depends on whether teach- 
ers share offices b) One-to-one assuming only one teacher per 
bus c) Most likely not one-to-one, especially if salary is set by 
a collective bargaining agreement d) One-to-one 19. An- 
swers will vary. a) Set of offices at the school; probably not 
onto b) Set of buses going on the trip; onto, assuming every 
bus gets a teacher chaperone c) Set of real numbers; not onto 
d) Set of strings of nine digits with hyphens after third and 
fifth digits; not onto 21. a) The function f(x) with f(x) = 
3x + 1whenx > Oand f(x) = -3x+2 when x <0 
b) f(x) = |x| +1 c) The function f(x) with fœ) = 2x + 1 
when x > 0 and f(x) = —2x when x < 0 d fœ) =x +1 
23. a) Yes b) No c) Yes d) No 25. Suppose that f is strictly 
decreasing. This means that f(x) > f(y) whenever x < y. To 
show that g is strictly increasing, suppose that x < y. Then 
g(x) = 1/f@) < 1/f@) = gy). Conversely, suppose that 
g is strictly increasing. This means that g(x) < g(y) when- 
ever x < y. To show that f is strictly decreasing, suppose that 
x < y. Then f(x) = 1/g(x) > 1/20) = f(y). 27. a) Let 
f be a given strictly decreasing function from R to itself. If 
a < b,then f(a) > f(b); ifa > b, then f(a) < f(b). 
Thus, if a # b, then f(a) # f(b). b) Answers will vary; for 
example, f(x) = 0 forx < Oand f(x) = —x forx > 0. 
29. The function is not one-to-one, so it is not invertible. On 
the restricted domain, the function is the identity function on 
the nonnegative real numbers, f(x) = x, so it is its own in- 
verse. 3l.a)f(S) = {0,1,3} b)f(S) = {0, 1, 3,5, 8} 
c) f(S) = {0, 8, 16, 40} d) f(S) = {1, 12, 33,65} 33. a) Let 
xandy be distinct elements of A. Because g is one-to-one, g(x) 
and g(y) are distinct elements of B. Because f is one-to-one, 
F(g(x)) = (fog)(x) and f(g(y)) = (fog)(y) are distinct elements 
of C. Hence, fog is one-to-one. b) Lety € C. Because f is 
onto, y = f(b) for some b € B. Now because g is onto, b = g(x) 
for some x € A. Hence, y = f(b) = f(g(x)) = (fog)(x). It 
follows that fog is onto. 35.LetA = {a}, B = {b,, by}, 
C = {c}, g(a) = b,, and f(b,) = f(b) = c. 37. No. For 
example, suppose that A = {a}, B = {b,c}, and C = {d}. 
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Let g(a) = b, f(b) = d,andf(c) = d. Then f and fog 
are onto, but g is not. 39.(f + g(x) = xX +x 4+ 3, 
FD) = 2 +2 +x4+2 41.f is one-to-one because 
fœ) =f) > ax, +b =ax +b > ax, = ax, > x =X. 
f is onto because f((y — b)/a) = y. fO) = Y — b)/a. 
43. a) A=B=R,S={x|x>0}, T={x|x< 0}, f@=xr 
b) It suffices to show that f(S) A f(T) C f(S A T). Let y € B 
be an element of f(S) N f(T). Then y € f(S), so y = f(x) for 
some x, € S. Similarly, y = f (x2) for some x, € T. Because f 
is one-to-one, it follows that x, = x3. Therefore, x, E SNT, 
soy Ef(SAT). 45.a) {x|0<x<1} b) {x|-1 <x< 2} 
AØ 47. f-\(S) = (x EA | f@) €S} = EAIA ES} 
= f-l(S) 49. Let x = |x] +e, where e is a real number with 
O0O<ec< life < L, then |x} -1I < x-i < |x], so 
[x - 1] = |x] and this is the integer closest to x. If e > L, 
then |x] < x- L < [x] + 1, so [x — sl = |x] + 1 and 
this is the integer closest to x. If € = L, then [x — I = |x], 
which is the smaller of the two integers that surround x and 
are the same distance from x. 51. Write the real number x 
as |x] +e, where e is areal number with 0 < e < 1. Because 
e = x — |x], it follows that O < —|x] < 1. The first two 
inequalities, x — 1 < |x] and |x] < x, follow directly. For the 
other two inequalities, write x = [x] — e’, where 0 < €’ < 1. 
Then 0 < [x] —x < 1, and the desired inequality follows. 
53. a) If x < n, because |x| < x, it follows that |x] < n. 


Suppose that x > n. By the definition of the floor function, 


it follows that |x] > n. This means that if |x] < n, then 
x <n. b)Ifn < x, then because x < [x], it follows that 
n < [x]. Suppose that n > x. By the definition of the ceiling 
function, it follows that [x] < n. This means that if n < [x], 
thenn <x. 55. Ifn is even, then n = 2k for some integer k. 
Thus, |n/2] = |k] = k = n/2. If nis odd, then n = 2k + 1 
for some integer k. Thus, |n/2] = |k + 1] = k = (n — 1)/2. 
57. Assume that x > 0. The left-hand side is [—x] and the 


right-hand side is — |x]. If xis an integer, then both sides equal 
—x. Otherwise, let x =n+e, where n is a natural number and e 


is areal number with 0 < e < 1. Then [~x] = [-n—-—e] = -n 
and —|x] = —|n+e] = -n also. When x < 0, the equa- 
tion also holds because it can be obtained by substituting —x 
forx. 59.[b] —|a]-—1 6l.a)1 b)3 ©) 126 d) 3600 
63.a) 100 b) 256 c) 1030 d) 30,200 


65. 44 eo 
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67. 34 —o 
a4 
1 — 
fpu) 
-2 -1 0 1 2 3 
+-1 
— -2 
—o -3 
69. a) 3 _ 
2+ —o 
1+e—o 
1 i aH i 
—4 -2 2 4 2 
o_o +-1 
— -2 
—- =3 
c) 3 ome d) 44 eo 
FL eee 3 — 
jiis 2 —— 
Eos] l 
-12 -9 -6 -3 3.6 9 12 
oe +-I ie] 1 
—— =I 
o— +-2 
— -2 
—_— +33 wo 1-3 
e) 4 . f) — 54L 
3 4 4 . 44 . 
24 . 37 = 
24 — 
1o——o 
1+ eo 
es ee eee ee ee eee 
3 4 1 2 to 
o—— -! 2 =l Ja ! 2 
. +-2 2 
— -3 des 
. -4 -4 


g) See part (a). 71. f1) = Q- 1) 73.a) fins @ = 
lexEANBexEAandx EB © fx) = 1 and fp@) = 
1 efaa =1 DfyygQ)=leoxeEAUBoxeAor 
XEBS fax) = 1 or fea) = 1 e fatra) -aA = 1 
Offy=lexEA e xgAefia)=0 e 1-fia)=1l 
Dfios®) =1exeEAPB © x E Aandx ¢ B) 
or (x é Aandx E B) © fx(x) +f — Afaa = 1 
75. a) True; because |x] is already an integer, [|x]] = [x]. 
b) False; x = 1 is a counterexample. c) True; if x or y is an 
integer, then by property 4b in Table 1, the difference is 0. If 
neither x nor y is an integer, then x = n +e andy = m + ô, 
where n and m are integers and € and 6 are positive real num- 
bers less than 1. Then m +n < x+y < m+n+4+2,5s0 
[x+y] is either m +n + 1 or m +n + 2. Therefore, the given 
expression is either (n + 1)+ (m+1)-(m+n+1)=1or 
(n+ 1)+(m+1)-— (m+n + 2) = 0, as desired. d) False; 
x = + and y = 3 is a counterexample. e) False; x = lis 
a counterexample. 77. a) If x is a positive integer, then the 
two sides are equal. So suppose that x = n? + m + e, where 
n? is the largest perfect square less than x, m is a nonnegative 
integer, and 0 < € < 1. Then both yx and y |x] = Vn? +m 
are between n and n + 1, so both sides equal n. b) If x is 
a positive integer, then the two sides are equal. So suppose 
that x = n? — m — e, where n? is the smallest perfect square 


greater than x, m is a nonnegative integer, and € is a real num- 
ber with 0 < e < 1. Then both yx and y [x] = yn? -m 
are between n — 1 and n. Therefore, both sides of the equa- 
tionequaln. 79. a) Domain is Z; codomain is R; domain of 
definition is the set of nonzero integers; the set of values for 
which f is undefined is {0}; not a total function. b) Domain is 
Z; codomain is Z; domain of definition is Z; set of values for 
which f is undefined is Ø; total function. c) Domain is Z x Z; 
codomain is Q; domain of definition is Z x (Z — {0}); set of 
values for which f is undefined is Z x {0}; not a total function. 
d) Domain is Z x Z; codomain is Z; domain of definition is 
ZXZ; set of values for which f is undefined is Ø; total function. 
e) Domain is Z x Z; codomain is Z; domain of definitions is 
{(m,n) | m > n}; set of values for which f is undefined is 
{(m,n) | m < n}; nota total function. 81. a) By defini- 
tion, to say that S has cardinality m is to say that S has exactly 
m distinct elements. Therefore we can assign the first object 
to 1, the second to 2, and so on. This provides the one-to-one 
correspondence. b) By part (a), there is a bijection f from S to 
{1, 2, ..., m} and a bijection g from T to {1, 2, ..., m}. Then 
the composition g7} of is the desired bijection from S to T. 


Section 2.4 


1.a)3 b)—1 c)787 d) 2639 3.a)a) = 2,a, = 3, 
da, =5, a,=9 b)aą = la, =4, a,=27, a; = 256 
c) ap 0, a, 0, ay 1, a, 1 d)a =0,a, = 1, 
ay = 2a = 3 5.a)2,5, 8, 11, 14, 17, 20, 23, 26, 29 
bdLL42223334 90L,L33557,7,9,9 
d)—-1, —2, —2, 8, 88, 656, 4912, 40064, 362368, 
3627776 ~—e) 3, 6, 12, 24, 48, 96, 192, 384, 768, 1536 
f) 2, 4, 6, 10, 16, 26, 42, 68, 110, 178 g) 1, 2, 2, 3, 3, 3, 3, 4, 
44 h)3, 3,5,44, 3,5,5,4,3  7.Each term could be 
twice the previous term; the nth term could be obtained from 
the previous term by adding n — 1; the terms could be the 
positive integers that are not multiples of 3; there are in- 
finitely many other possibilities. 9. a) 2, 12, 72, 432, 2592 
b) 2, 4, 16, 256, 65, 536 œ) 1, 2,5, 11,26 d) 1, 1, 6, 27, 204 
e) 1,2,0,1,3 11. a) 6,17, 49, 143,421 b) 49 =5-17-6-6, 
143 =5-49-6-17,421 = 5- 143- 6-49 © 5a,ı-— 
6a, =5(2"7! + 5-3"-!) — 6(2"-? + 5-3"-2) = 2"-2(10-6) + 
3"(75 — 30) = 2"? . 4 + 372 .9.5=2" +3" .5=a, 
13.a) Yes b)No c)No d)Yes e) Yes f)Yes g)No 
h)No 15.a)a,_, + 2a, 2 +2n-9 = -(n- 1) + 2+ 
2[-(n — 2) +2] + 2n- 9 = -n+2 = a, b)a,ı + 
2a,» +2n — 9 = 5(-1)"! — (n — 1) + 2 + 2[5(-1)"? — 
(n — 2) +2] + 2n — 9 = 5(-1)} 7 ?(-1 +2) -n +2 =a, 
c) a,_; + 2a,_5 +2n — 9 =3(-1)""'42""! — (n — 1)+2 + 
28-17 + 2 — (n - 2) + 2] + 2n- 9 = 3(-1)"? 
(-l + 2) + 2"°°(2 4+ 2)-n+2 = a, da, + 
2a, +2n-9 = 7-2"! — (n—1) + 2 + 27-277 - 
(n — 2) + 2]4+2n-9 = 2"°(7-242-7)—-n + 2=a, 
17.a)a, =2-3" b)a, =2n+3 cha, = 1+n(n+ 1)/2 
dja, =w+4n+4 ea, = 1 fja, = 3B"! - 1/2 
g)a, =5n! hha, =2"n! 19.a)a,=3a,_, b) 5,904,900 
2l.a)a,=n+a,_),d)9 =0 b) a,j. =78 c)a, =n(n+ 1)/2 


23. B(k) = [1 + (0.07/12)] BK — 1) — 100, with B(0) = 5000 
25. a) One 1 and one 0, followed by two 1s and two Os, fol- 
lowed by three 1s and three Os, and so on; 1, 1, 1 b) The 
positive integers are listed in increasing order with each 
even positive integer listed twice; 9, 10, 10. c) The terms 
in odd-numbered locations are the successive powers of 2; 
the terms in even-numbered locations are all 0; 32, 0, 64. 
d) a, = 3 -2”"'; 384, 768, 1536 e)a, = 15 — 7(n — 1) = 
22 — Tn; —34, -41,-48 fla, = (n? + n + 4)/2; 57, 68, 
80 g)a, = 2n?; 1024, 1458, 2000 h) d, =n! + 1; 362881, 
3628801, 39916801 27. Among the integers 1, 2, ..., a, 
where a, is the nth positive integer not a perfect square, the 
nonsquares are 44, dy, ..., a,, and the squares are 17329 oe Rs 
where kis the integer with k? < n+k < (k+1)*. Consequently, 
a, = n + k, where k? < a, < (k + 1}. To find k, first note 
that k? <n+k<(k+1)?,soP?+1<nt+k<(k+1¥—-1. 
Hence, (k= +) +3 =K -k+1 <n< kK +k=(k++}- i. 
It follows that k — + < 4/n < k + Ł, sok = {yn} and 
ap=n+k=n+ {yn}. 29.a)20 b)11 ¢)30 d)511 
31.a) 1533 b)510 œc)4923 d)9842 33.a)21 b)78 
c)18 d)18 35. G —4_\) = ap — a 37.a) n? 
b)n(n + 1)/2 39.15150 41.34320 43, IBOD 4 
MD + (n+ (mn — (n +1) + 1),wheren = | ym] — 1 
45.a)0 b)1680 c)1 d)1024 47.34 


Section 2.5 


1. a) Countably infinite, —1, —2, —3, —4, ... b) Countably 
infinite, 0, 2, —2, 4, —4, c) Countably infinite, 
99, 98, 97, ... d) Uncountable e) Finite f) Countably infi- 
nite, 0, 7, —7, 14, —14, ... 3. a) Countable: match n with 
the string of n 1s. b) Countable. To find a correspondence, 
follow the path in Example 4, but omit fractions in the top 
three rows (as well as continuing to omit fractions not in low- 
est terms). c) Uncountable d) Uncountable 5. Suppose m 
new guests arrive at the fully occupied hotel. Move the guest 
in Room n to Room m + n for n = 1, 2, 3, ...; then the new 
guests can occupy rooms | tom. 7.Forn = 1, 2,3, ..., put 
the guest currently in Room 2n into Room n, and the guest 
currently in Room 2n — 1 into Room n of the new build- 
ing. 9. Move the guest currently in Room i to Room 2i + 1 
fori = 1, 2,3,.... Put the jth guest from the kth bus into 
Room 2¥(2j + 1). 11. a)A = [1,2] (closed interval of 
real numbers from | to 2), B = [3,4] b)A = [1, 2] U Z*, 
B = [5,4 uU Zt oA = [1,3], B = [2,4] 13. Suppose 
that A is countable. Then either A has cardinality n for some 
nonnegative integer n, in which case there is a one-to-one 
function from A to a subset of Z* (the range is the first n pos- 
itive integers), or there exists a one-to-one correspondence f 
from A to Z+; in either case we have satisfied Definition 2. 
Conversely, suppose that |A| < |Z*|. By definition, this 
means that there is a one-to-one function from A to Zt, so A 
has the same cardinality as a subset of Z+ (namely the range 
of that function). By Exercise 16 we conclude that A is count- 
able. 15. Assume that B is countable. Then the elements of 
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B can be listed as b}, b), b3,.... Because A is a subset of B, 
taking the subsequence of {b,,} that contains the terms that 
are in A gives a listing of the elements of A. Because A is 
uncountable, this is impossible. 17. Assume that A — B is 
countable. Then, because A = (A — B) U (A N B), the elements 
of A can be listed in a sequence by alternating elements of 
A — B and elements of A N B. This contradicts the uncount- 
ability of A. 19. We are given bijections f from A to B and g 
from C to D. Then the function from A x C to B XD that sends 
(a, c) to (f(a), g(c)) is a bijection. 21. By the definition of 
|A| < |B], there is a one-to-one function f : A > B. Similarly, 
there is a one-to-one function g : B > C. By Exercise 33 
in Section 2.3, the composition gof : A —> C is one-to-one. 
Therefore, by definition |A| < |C|. 23. Using the Axiom 
of Choice from set theory, choose distinct elements a,, az, 
az, ...of A one at a time (this is possible because A is infi- 
nite). The resulting set {a,, dy, az, ...} is the desired infinite 
subset of A. 25. The set of finite strings of characters over 
a finite alphabet is countably infinite, because we can list 
these strings in alphabetical order by length. Therefore, the 
infinite set S can be identified with an infinite subset of this 
countable set, which by Exercise 16 is also countably infinite. 
27. Suppose that A,, Ay, A3, ... are countable sets. Because 
A; is countable, we can list its elements in a sequence as 
dj}, Gj, aj, -.. . The elements of the set )_, A; can be listed 
by listing all terms a; with i + j = 2, then all terms a; 
with i + j = 3, then all terms aj with i + j = 4, and so 
on. 29. There are a finite number of bit strings of length m, 
namely, 2”. The set of all bit strings is the union of the sets 
of bit strings of length m for m = 0, 1, 2, .... Because the 
union of a countable number of countable sets is countable 
(see Exercise 27), there are a countable number of bit strings. 
31. It is clear from the formula that the range of values the 
function takes on for a fixed value of m + n, say m + n = x, is 
(x — 2)(x — 1)/2 + 1 through (x — 2)(x — 1)/2 + (x — 1), be- 
cause m can assume the values 1, 2, 3, ..., (x — 1) under these 
conditions, and the first term in the formula is a fixed positive 
integer when m + n is fixed. To show that this function is 
one-to-one and onto, we merely need to show that the range 
of values for x + 1 picks up precisely where the range of 
values for x left off, i.e., that f(x — 1, 1) +1 =f, x). We have 


fa@-Lit+l CDAD + (x= 1) +1 ee ie ne 


2 

ou +1 =f(,x). 33. By the Schréder-Bernstein theo- 
rem, it suffices to find one-to-one functions f : (0, 1) > [0, 1] 
and g : [0,1] > (0, 1). Let f(x) = x and g(x) = (x + 1)/3. 
35. Each element A of the power set of the set of positive 
integers (i.e, A C Z*) can be represented uniquely by the 
bit string a,a,a3..., wherea; = lifi E€ Aanda,; = 0 
if i ¢ A. Assume there were a one-to-one correspondence 
f: Z* > P(Z*). Form a new bit string s = 5,553 ... by set- 
ting s; to be 1 minus the ith bit of f (i). Then because s differs 
in the i bit from f (i), s is not in the range of f, a contradiction. 
37. For any finite alphabet there are a finite number of strings 
of length n, whenever n is a positive integer. It follows by the 
result of Exercise 27 that there are only a countable number 
of strings from any given finite alphabet. Because the set of 
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all computer programs in a particular language is a subset of 
the set of all strings of a finite alphabet, which is a count- 
able set by the result from Exercise 16, it is itself a countable 
set. 39. Exercise 37 shows that there are only a countable 
number of computer programs. Consequently, there are only 
a countable number of computable functions. Because, as 
Exercise 38 shows, there are an uncountable number of func- 
tions, not all functions are computable. 41. a) Note that if 
x is in the chain generated by y, then by the way the chains 
are generated, y is in the chain generated by x, so these two 
chains are the same. Thus, if x is in both the chain generated 
by y, and the chain generated by y,, then the chain generated 
by y, and the chain generated by y, are both the same as the 
chain generated by x and are therefore the same chain. b) A 
moment’s reflection will show that by the way the chains 
are constructed, this is true. c) Again, this is clear from the 
construction. d) Because the chains are disjoint and every 
element of A appears in exactly one chain and every element 
of B appears in exactly one chain, the function / cannot map 
two different elements of A to the same element of B. e) If an 
element b of B appears in a chain of types 1, 2, or 3, then it is 
preceded by an element of A, which maps to it under h. If b 
appears in a chain of type 4, then it is followed by an element 
of A, which maps to it under A. 


Section 2.6 
1 
l.a)3x4 b)|4 [2 0 4 6 d) 1 
3 
oli 2 1 3a ji 11 b)|2 -2 -3 
1 0 1 2 18 1 0 2 
1 4 3 9 -4 4 
3-6 7 


Of4 15 4 1] 5 | 9/5 | 

-3 10 2 -3 -1/5 4/5 

0 2 -8 6 

1 -8 18 -13 

7.0+A=[0+a,] = [a,+0] =0+A 9.A+B+O)= 
[ay + (by +¢,)] = [Cay +b) + cy] =(A+B)+C 11. The 
number of rows of A equals the number of columns of B, and 
the number of columns of A equals the number of rows of B. 


13. ABC) = [Zug (Zpberen) | = [EE erea] 
[ZDetebecn| = |E (Zete) = ame 


l n 
0 1 
A+B = [a; + bj]. We have (A + By = la; + Bj] = [ay] + 
[b;;] = A' + B’. b) Using the same notation as in part (a), we 
have B'A’ = 2, bait. = be ay = (AB)’, because 
the (i, /)th entry is the (j, i)th entry of AB. 19. The result 


15. A” = | 17. a) Let A = [aj] and B = [b;]. Then 


b| |d  —b|_|ad-bc 0 _ 

| ie el 0 sd | ~ 
d —b| ja b 
— a c al 
A(A --(A(AAT))AT!) -- ATHA! by the associative law. 
Because AA! =I, working from the inside shows that 
A"(A7!)" = I. Similarly (A7!)"A” = I. Therefore, (A”)7! = 
(A7')". 23. The (i j)th entry of A + A’ is a, + aj, which 
equals aji + ajj the (j, ith entry of A + A‘, so by definition 
A + Aʻ is symmetric. 25. x1 = 1, x, = —1, x, = —2 


follows because f 


(ad — bòl, = 21. A”(A-7!)” 


ll 


27.1; 1 ıl Dio o 1] Of1 1 
f i i 1 0 0 1 1 
1 0 0 0 1 1 0 
2.aji 0 of Dji o of f1 0 o 
1 1 0 1 0 1 1 1 
1 0 1 1 1 0 i 1 


31. a) AVB = lay V bil = [by Va; =BvA bDAAB = 
la; A bj) = [by Aaj] =BAA 33.a) AV (B A C) = 
[al Vv [bi A cy] = la; Vv (bj A cy) = [aj Vv bi) A 
(ajVcj)1 = [a;Vb;] AlajVcjl = (AVB)A(AVC) b) AA(BYV 
C)= [a;l Alb; y cy] = [ajA (b; vcy) = [(ajAbiy)V (ajnc) = 
la Ab] V [aj Ac] =(AAB)V(AAC) 35.A0(BOC) 


[Vitun (Ve (Bard €x))) = [Vp Vlan A Bur cn)| 


V Vg laig ^bgr acn] = LV. (Vi (aig Abg) ) ^ cal 
(AOB)OC 


ll 


ll 


ll 


Supplementary Exercises 


l.a)A bD)ANB QA-B AANB AB 
3.Yes 5.A4—(A-B) =A-(ANB)=AN (ANB) = An 
(AU B)=(ANA)U(ANB)=GBU(ANB)=ANB_ 7. Let 
A = {1}, B = Ø, C = {1}. Then (A — B) —C = Ø, but 
A — (B — C) = {1}. 9.No. For example, let A = B = 
{a,b}, C = Ø, and D = {a}. Then (A — B) — (C — D) = 
Ø — Ø = Ø, but (A — C) — (B — D) = {a,b} — {b} = {a}. 
11. a) |Ø] < |AN Bl < |A] < [AUB < |U] b)|Ø| < 
|A — B| < |A ® B| < |AUB|<|A|+ |B| 13.a) Yes, no 
b) Yes,no c)f has inverse with f-'(a) = 3, f-'(b) = 4, 
fol@ = 2,f-'@=1; ghas no inverse. 15. If f is one-to- 
one, then f provides a bijection between S and f(S), so they 
have the same cardinality. If f is not one-to-one, then there ex- 
ist elements x and y in S such that f(x) = f(y). Let S = {x, y}. 
Then |S| = 2 but [f(S)| = 1. 17.Letx € A. Then 
S-({x}) = {f0) | y E {x}} = {f(x)}. By the same reasoning, 
S(x} = {g(x)}. Because Sp = S,, we can conclude that 
fœ} = {g()}, and so necessarily f(x) = g(x). 19. The 
equation is true if and only if the sum of the fractional parts 
of x and y is less than 1. 21. The equation is true if and only 
if either both x and y are integers, or x is not an integer but the 
sum of the fractional parts of x and y is less than or equal to 1. 
23. If x is an integer, then |x] + |m — x] =x+m-x= m. 
Otherwise, write x in terms of its integer and fractional parts: 
x =n + e€, where n = |x] and0 < e < 1. In this case |x] + 


[m—-x|] =|n+e]+|m—-—n-e] =n+m—-n-1l=m-1. 
25. Write n = 2k+1 for some integer k. Then n? = 4k? +4k+1, 
son?/4 =K +k+ L, Therefore, [n?/4] = k? + k + 1. But 
(n? +3)/4 = (4k? + 4k +1 +3)/4= k +k41. 27. Let 
x = n + (r/m) + e, where n is an integer, r is a nonnegative 
integer less than m, and e is a real number with 0 < e < 1/m. 
The left-hand side is |nm + r + me| = nm + r. On the right- 
hand side, the terms |x] through |x + (m +r — 1)/m] are all 
just n and the terms from |x + (m — r)/m] on are all n + 1. 
Therefore, the right-hand side is (m—r)n+r(n+1)=nm+r, 


as well. 29.101 3l.a,; = 1; @,,, = n- a for all 
n > 0; anda, = n+ a, for alln > 0. The next 
four terms are 5346, 5353, 37, 471, and 37,479. 33. If each 


f7'G) is countable, then S = f7'(1) u f-'(2) U + is the 
countable union of countable sets and is therefore countable 
by Exercise 27 in Section 2.5. 35. Because there is a one- 
to-one correspondence between R and the open interval (0, 1) 
(given by f(x) = 2 arctan(x)/z), it suffices to shows that 
I(0, 1) x (0, 1)| = |(O, 1)|. By the Schréder-Bernstein theorem 
it suffices to find injective functions f : (0, 1) > (0, 1) x (0, 1) 
and g : (0, 1) x (0, 1) > (0, 1). Let f(x) = 2). For g we 
follow the hint. Suppose (x, y) € (0, 1) x (0, 1), and represent 
x and y with their decimal expansions x = 0.x,x x3... and 
y =0.y,y2y3 -.-, never choosing the expansion of any number 
that ends in an infinite string of 9s. Let g(x, y) be the decimal 


expansion obtained by interweaving these two strings, namely 
- 1 0 2 0 1 
Osi. BAY = b | ATH = L ol s 
-1 0 i 0 -=i 
äni 4n+3 — 
A = i afa = f gj] -forn > 0 
a b O 1 
39. Suppose that A = i a . Le B = k | . Be- 
cause AB = BA, it follows that c = 0O anda = d. Let 
B = k o . Because AB = BA, it follows that b = 0. 
Hence, A = | O| Zal. 41.a) Let A O0 = [b]. Then 


bj = (ai A0) V + V (a AO) = 0. Hence, A ©0 = 0. Similarly, 
00A =0. b) AVO = |a; v 0] = [a;] = A. Hence, A v0 = A. 
Similarly, 0V A = A. © AAO = |a; A0] = [0] = 0. Hence, 
A A0= 0. Similarly, 0A A = 0. 


CHAPTER 3 


Section 3.1 


1. max := 1,i := 2 max := 8a := 3, max := 12,1 := 4, 
(3=5,1:=6,0:= 7, max = 14,73= 8,1 = 9,7:= 10, 0:= 11 
3. procedure AddUp(q,, ... 
sum := a, 
fori: =2ton 
sum := sum + a; 
return sum 


, a: integers) 
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procedure duplicates(a,, az, ..., 
nondecreasing order) 
k := 0 {this counts the duplicates } 


eo 


fe 


9. procedure palindrome check(a, a, ... a} string) 


11 


The minimum number of assignments needed is three. 


13 


while j < n 
if a; = a; then 
k:=k+1 
Cy = aj 
while j < n and a; = cy 
j:=j+1 
j:=j+1 
{ci Cz ..., Cp is the desired list} 
procedure last even location(a;,az, ... 
k:=0 
for i := 1 ton 
if a; is even then k := i 
return k {k = 0 if there are no evens} 


answer := true 
for i := 1 to |n/2] 
if a; # a,,,_; then answer := false 
return answer 


. procedure interchange(x, y: real numbers) 
z=% 
x:=y 
y:=z 


a,; integers in 


,d,, integers) 
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. Linear search: i := 1,7 := 2, i := 3, i := 4,7: 


6, 


i := 7, location := 7; binary search: i := 1, j := 8, m := 4, 


t: 


5, m := 6, i := 7, m := 7, j := 7, location := 7 


15 


17. 


19. 


. procedure insert(x, a), do, ..., a„: integers) 
{the list is in order: a, < a, < + < a} 
an = X41 
i= l 
while x > a; 

i:=it+1 
forj :=Oton—-i 
an-j+1 = an-j 

ü= 

{x has been inserted into correct position } 

procedure first largest(a,, ... 

max := ay 


, a, integers) 


location := 1 
for i := 2 ton 
if max < a; then 
max := d; 
location := i 
return location 


mean := (a +b + ¢)/ 3 


{the six different orderings of a, b, c with respect 


to > will be handled separately } 


procedure mean-median-max-min(a, b, c: integers) 
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if a > b then 
if b > c then median := b; max := a;min := c 


(The rest of the algorithm is similar.) 


21. procedure first-three(a,, a, ..., d,: integers) 
if a, > a, then interchange a, and a, 
if a, > a, then interchange a, and a; 
if a, > a, then interchange a, and a, 
23. procedure onto(f: function from A to B, where 
A= {a,,...,4,},B = {bis e; bm} äp ..54 


n? 


bi, ...,b,, are integers) 
for i := 1 tom 
hit(b;) := 0 
count := 0 
for j := 1 ton 


if hit(f(a)) = 0 then 
hit(f(a)) = 1 
count := count + 1 
if count = m then return true else return false 


25. procedure ones(a: bit string, a = a,d, ...d,) 


count:= 0 
for i := 1 ton 
if a; := 1 then 


count := count + 1 
return count 


27. procedure ternary search(s: integer, a,,d, ..., ap: 

increasing integers) 

i:=1 

ji=n 

while i < j — 1 
1:= |é+)/3] 
u:= [20+ )/3] 
if x >a, theni:=u+1 
else if x > a, then 


i:=14+1 
ji=u 
else j := l 


if x = a; then location := i 
else if x = a; then location := j 
else location := 0 


return location {0 if not found} 


29. procedure find a mode(a,, dy, ..., a„: nondecreasing 
integers) 
modecount := 0 
i= 1 
while i < n 
value := a; 
count := 1 
while i < n and a; = value 
count := count + 1 
i:=i+1 
if count > modecount then 
modecount := count 
mode := value 
return mode 


31. Assume the input is strings a,a,...a, and b,b)...b,, 
where each character is a letter, A through Z. Assume also 
that a function index is available, such that index(x) is the 
position of the letter x in the alphabet (index(‘A’) = 1,..., 
index(‘Z’) = 26). a) Initialize a-count and b-count to be 
lists of length 26 with all values equal to 0. For i from 1 
to n increment a-count(index(a;)) and b-count(index(b,)). If 
a-count(i) = b-count(i) for all i from 1 to 26, then return 
“true”; otherwise return “false.” b) Sort both strings into al- 
phabetical order. Then the two original strings were anagrams 
if and only if the sorted strings are identical. 
33. procedure find duplicate(a,, dy, ..., a„: integers) 
location := 0 
Va 2 
while i < n and location = 0 
j=l 
while j < i and location = 0 
if a; = a; then location := i 
else j :=j+1 
i:=i+1 
return location 
{location is the subscript of the first value that 
repeats a previous value in the sequence } 


35. procedure find decrease(a,, a, ..., d,;; positive 
integers) 

location := 0 

i:=2 

while i < n and location = 0 
if a; < a;_, then location := i 
else i :=i+ 1 

return location 

{location is the subscript of the first value less than 
the immediately preceding one} 


37. At the end of the first pass: 1, 3, 5, 4, 7; at the end of the 
second pass: 1, 3, 4, 5, 7; at the end of the third pass: 1, 3, 4, 
5, 7; at the end of the fourth pass: 1, 3, 4, 5, 7 


39. procedure better bubblesort(a}, ..., a,: integers) 
i: = 1; done : = false 
while i < n and done = false 
done : = true 
forj:=1ton—-i 
if a; > a;,, then 
interchange a, and a;,, 
done : = false 
i:=i+1 
{a,, ..., @, is in increasing order} 
41. At the end of the first, second, and third passes: 1, 3, 5, 7, 4; 
at the end of the fourth pass: 1, 3,4,5,7 43. a) 1,5, 4, 3, 2; 
1, 2, 4, 3, 5; 1, 2, 3, 4, 5; 1, 2, 3,4,5  b)1, 4, 3, 2,5; 1 
2,3, 4,5; 1, 2, 3, 4,5; 1, 2,3,4,5 ©) 1, 2,3, 4, 5; 1, 2, 3, 
4,5; 1, 2, 3,4, 5; 1,2,3,4,5 45. We carry out the linear 
search algorithm given as Algorithm 2 in this section, except 
that we replace x # a; by x < a and we replace the else 
clause with else location :=n+ 1. 47.24+34+4+--4+n= 
(n?4+n—2)/2 49. Find the location for the 2 in the list 3 (one 


> 


comparison), and insert it in front of the 3, so the list now 
reads 2, 3, 4, 5, 1, 6. Find the location for the 4 (compare it 
to the 2 and then the 3), and insert it, leaving 2, 3, 4, 5, 1, 6. 
Find the location for the 5 (compare it to the 3 and then the 
4), and insert it, leaving 2, 3, 4, 5, 1, 6. Find the location for 
the 1 (compare it to the 3 and then the 2 and then the 2 again), 
and insert it, leaving 1, 2, 3, 4, 5, 6. Find the location for the 6 
(compare it to the 3 and then the 4 and then the 5), and insert 
it, giving the final answer 1, 2, 3, 4, 5, 6. 


51. procedure binary insertion sort(a,, dy, ..., Ay: 
real numbers with n > 2) 
for j :=2ton 
{binary search for insertion location i} 
left := 1 
right :=j—1 
while left < right 
middle := | (left + right)/2| 
if aj > amiddie then left := middle + 1 
else right := middle 
if a; < aip then i := left else i := left + 1 
{insert a; in location i by moving a; through a;_; 
toward back of list} 


m := a, 
for k := 0toj—i— 1 
aik = Ajk- 
a; :=m 
{a}, a, ..., a, are sorted} 


53. The variation from Exercise 52 55.m=3,n=8;s =0, 
j = 1,no match; s = 1,7 = 1,7 = 2,7 = 3, no match; s = 2, 
j = 1,no match; s = 3,7 = 1,7 = 2, no match; s = 4, 
j = 1, no match; s Sj lj 2J 3-1 4, 
“5 is a valid shift” 57. a) Two quarters, one penny b) Two 
quarters, one dime, one nickel, four pennies c) Three quar- 
ters, one penny d) Two quarters, one dime 59. Cashier’s 
algorithm uses fewest coins in parts (a), (c), and (d). a) Two 
quarters, one penny b) Two quarters, one dime, nine pen- 
nies c) Three quarters, one penny d) Two quarters, one dime 
61. The 9:00-9:45 talk, the 9:50-10:15 talk, the 10:15-10:45 
talk, the 11:00-11:15 talk 63. a) Order the talks by start- 
ing time. Number the lecture halls 1, 2, 3, and so on. For each 
talk, assign it to lowest numbered lecture hall that is currently 
available. b) If this algorithm uses n lecture halls, then at the 
point the nth hall was first assigned, it had to be used (other- 
wise a lower-numbered hall would have been assigned), which 
means that n talks were going on simultaneously (this talk 
just assigned and the n — 1 talks currently in halls 1 through 
n—1). 65. Here we assume that the men are the suitors and 
the women the suitees. 
procedure stable(M,, M,,..., 

preference lists) 
for i := 1 tos 

mark man i as rejected 
for i:= 1 tos 

set man i’s rejection list to be empty 
for j := 1 tos 

set woman j’s proposal list to be empty 


M, Wi, Wo, ..., W; 
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while rejected men remain 
for i := 1 tos 
if man i is marked rejected then add i to the 
proposal list for the woman j who ranks highest 
on his preference list but does not appear on his 
rejection list, and mark i as not rejected 
for j := 1 tos 
if woman j’s proposal list is nonempty then 
remove from j’s proposal list all men i 
except the man i) who ranks highest on her 
preference list, and for each such man ij mark 
him as rejected and add j to his rejection list 
for j := 1 to s 
match j with the one man on j’s proposal list 
{This matching is stable. } 


67. If the assignment is not stable, then there is a man m and a 
woman w such that m prefers w to the woman w’ with whom 
he is matched, and w prefers m to the man with whom she is 
matched. But m must have proposed to w before he proposed 
to w’, because he prefers the former. Because m did not end 
up matched with w, she must have rejected him. Women reject 
a suitor only when they get a better proposal, and they even- 
tually get matched with a pending suitor, so the woman with 
whom w is matched must be better in her eyes than m, con- 
tradicting our original assumption. Therefore, the marriage is 
stable. 69. a) (adapted from Wikipedia) Let m be the ma- 
jority element. Let n be a number defined at any step of the 
algorithm to be either the counter, if the majority candidate 
is m, or the negative of the counter otherwise. Then at each 
step in which the algorithm encounters a value equal to m, 
the value of n will increase by 1, and at each step at which 
it encounters a different value, the value of n may either in- 
crease or decrease by one. If m truly is the majority element, 
there will be more increases than decreases, and n will be pos- 
itive at the end of the algorithm. But this can be true only when 
the final stored element is m, the majority element. b) A coun- 
terexample is ABABC. 71. Run the two programs on their 
inputs concurrently and report which one halts. 


Section 3.2 


1. The choices of C and k are not unique. a) C = 1,k = 10 
b) C=4,k=7 ce) No d) C=5,k=1 e C=1,k=0 
f)C=1,k=2 3.x +9 44x47 < 4x‘ for all x > 9; 
witnesses C =4,k=9 5. (x?+1)/(x4-1) = x—1+2/(x+1) < 
x for all x > 1; witnesses C = 1,k = 1 7. The choices of 
C and k are not unique. a)n = 3,C = 3,k = 1 b)n =3, 
C=4,k=1 oc)n=1,C=2,k=1 d)n=0,C=2,k=1 
9.x? +4x + 17 < 3x3 for all x > 17, so x? + 4x + 17 is OQ), 
with witnesses C = 3, k = 17. However, if x? were 
O(x? + 4x + 17), then x? < CQ? + 4x + 17) < 3Cx for 
some C, for all sufficiently large x, which implies that x < 3C 
for all sufficiently large x, which is impossible. Hence, x° is 
not Ox? + 4x + 17). 11.3x44+1 < 4x4 = 8004/2) for 
allx > 1, so 3x* + 1 is O(x4 /2), with witnesses C = 8, 
k = 1. Also xł/2 < 3x4 +1 forall x > 0, so x4/2 is 
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O(3x* + 1), with witnesses C = 1,k = 0. 13. Because 
2” < 3” for all n > O, it follows that 2” is O(3"), with wit- 
nesses C = 1,k = 0. However, if 3” were O(2”), then for 
some C, 3” < C - 2” for all sufficiently large n. This says 
that C > (3/2)" for all sufficiently large n, which is impos- 
sible. Hence, 3” is not O(2”). 15. All functions for which 
there exist real numbers k and C with |f(x)| < C for x > k. 
These are the functions f(x) that are bounded for all suffi- 
ciently large x. 17. There are constants C}, C3, k,, and k, 
such that |f(x)| < C,|g@)| for all x > kı and |g(x)| < C,|h@)| 
for all x > k,. Hence, for x > max(k,, ky) it follows 
that FŒ] < C)|g@)| < C)C,|h@)|. This shows that f(x) is 
O(A(x)). 19. 2”! is O(2"); 27" is not. 21. 1000 log n; 
yn; n log n; n? / 1,000,000; 2”; 3"; 2n! 23. The algo- 
rithm that uses n log n operations 25. a) O(n?) b) O(n’) 
¢) O(n -n!) 27.a) O(n logn) b) O(n? (log n)*) ©) On”) 
29. a) Neither @(27) nor Q(x?) b) OQ?) and Q(x’) c) Neither 
OQ?) nor QQ?) d) Q(x’), but not OQ?) e) Q(x”), but not 
OQ?) f) Q(x?) and OQ?) 31. If f(x) is O(g(x)), then there 
exist constants C, and C, with Gilel < Fœ < Chlg@)|. 
It follows that Fœ] < Glew) and |e] < A/C) for 
x> k. Thus, f(x) is O(g(x)) and g(x) is O(f(x)). Conversely, 
suppose that f(x) is O(g(x)) and g(x) is O(f(x)). Then there 
are constants C}, Cj, k,, and k, such that [fœ] < C,|g@)| 
forx > k and |g(x)| < GF| forx > k,. We can as- 
sume that C, > 0 (we can always make C, larger). Then we 
have (1/C,)|g@| < Fl < Clg] for x > max(k,, kə). 
Hence, f(x) is O(g(@x)). 33. If f(x) is O(g(x)), then f(x) is 
both O(g(x)) and Q(g(x)). Hence, there are positive constants 
Ci, kı, Cy, and k, such that |f(x)| < C,|g(x)| for all x > k, 
and fœ] > C,|g(x)| for all x > kı. It follows that 


Cile@| < Fl < Cylg@]| whenever x > k, where 
k = max(k,,ky). Conversely, if there are positive constants C}, 
Cz, and k such that C,;|g(x)| < [f@)| < Cilg@)| for x > k, 
then taking ky = k, = k shows that f(x) is both O(g(x)) 


and ©(g(x)). 
35. 9 


k x 


37. If f(x) is OC), then |f(x)| is bounded between posi- 
tive constants C, and C,. In other words, f(x) cannot grow 
larger than a fixed bound or smaller than the negative of this 
bound and must not get closer to 0 than some fixed bound. 
39. Because f(x) is O(g(x)), there are constants C and k such 
that Fœ] < Clg()| for x > k. Hence, |f"@)| < C”|g"(x)| 
for x > k, so f"(x) is O(g"(x)) by taking the constant to be C”. 
41. Because f(x) and g(x) are increasing and unbounded, we 
can assume f(x) > 1 and g(x) > 1 for sufficiently large x. 


There are constants C and k with f(x) < Cg(x) forx > k. 
This implies that log f(x) < log C + log g(x) < 2 log g(x) for 
sufficiently large x. Hence, log f(x) is O(log g(x)). 43. By 
definition there are positive constraints C}, Ci; Cis Ci, kı, 
ki, ky, and k) such that A@) > Cile] for allx > k, 
fœ < C lgl for all x > k, 600) = Clew] for all x > kz, 
and f,(x) < C|g(x)| for all x > k}. Adding the first and third 
inequalities shows that f;(x) + fo(x) > (Ci + Cy)|g@)| for 
allx > k where k = max(k,, k,). Adding the second and 
fourth inequalities shows that f(x) +h) < (C1 + C))|g@)| 
for all x > k’ where k’ = max(kj, k}). Hence, f,(x) + fŒ) is 
O©(g(x)). This is no longer true if f, and f, can assume negative 
values. 45. This is false. Let f} = xX +2x, ha) = x? +x, and 
g(x) =x. Then f,(x) and f,(x) are both O(g(x)), but Fi -fE 
isnot. 47. Take f(n) to be the function with f(n) = nifn 
is an odd positive integer and f(n) = 1 if n is an even pos- 
itive integer and g(n) to be the function with g(n) = 1 ifn 
is an odd positive integer and g(n) = n if n is an even pos- 
itive integer. 49. There are positive constants C}, C), Ci: 
Ch, ky, ki, ky, and k, such that |A@| > C;|g)@)| for all 
x> ky Al Clg) for all x > k, KOO! > Clea 
for all x > k,, and |f(x)| < C)|g2()| for all x > k}. Be- 
cause f) and g, are never zero, the last two inequalities can 
be rewritten as |1/f,(x)| < (1/C,)|1/g.(x)| for all x > k, 
and |1/f,@)| 2 C/C))|1/g20)| for all x > k}. Multi- 
plying the first and rewritten fourth inequalities shows that 
ADOI > (C,/CL)1g1@)/g2(0)| for all x > max(k,, kK), 
and multiplying the second and rewritten third inequali- 
ties gives [f,@/f@)| < (C,/C)lg,)/g2@)| for all x > 
max(k', ky). It follows that f,/f, is big-Theta of g;/g2. 
51. There exist positive constants C}, C, kj, ky, ki, ky such 
that fœ, y) <C,lg@ y)| for all x>k, and y>k, and 
Fæ y)| = Glg y)| for all x > ki andy >k}. 53. (0? + 
xy + xlogy < Gxy3) = 27x%y3 forx > 1 and 
y>1, because x? < xy, xy < xy, and xlogy < xy. 
Hence, (x? + xy + xlogy)? is O(x°y*). 55. For all positive 
real numbers x and y, |xy] < xy. Hence, |xy] is O(xy) from 
the definition, taking C= 1 and k; =k, =0. 57. Clearly 
n? < n° foralln > 2; therefore, n? is O(n‘). The ratio 
n¢/n° = n*~ is unbounded so there is no constant C such 
that n? < Cn for large n. 59. If f and g are positive- 
valued functions such that lim, _,,, f(@x)/g(~) = C < œ, 
then f(x) < (C + 1)g(x) for large enough x, so f(n) is O(g(n)). 
If that limit is oo, then f(n) is not O(g(n)). Here repeated ap- 
plications of L’H6pital’s rule shows that lim,_,,, x“/b* = 0 
and lim,_,,, b*/x4 = œ. 61. To show that c” is O(n!), as- 
sume WLOG that c is an integer greater than 1. Claim that if 
n > +c, thenn! > c”. Both n! and c” have n factors. Replac- 
ing each of the factors from c? + 1 to c? + c in n! by c? only 
makes the product smaller. But then each of these factors c? 
can be factored as c - c and one of those factors of c moved to 
pair with the factors 1 through c in n!. At this point every fac- 
tor of n! is greater than or equal to c, so the product is greater 
than or equal to c”. To show that n! is not O(c”), look at the ra- 
tion!/c” and write this as (c!/c®)-((c+1)/c)-((c+2)/c)-((c+ 
3)/c) ++» ((2c)/c)-((2c+1)/c) -+ (n/c). Treat the product of the 


first c+ 1 factors as a constant and note that every other factor 
is greater than 2. Thus, the product is greater than an arbitrar- 
ily large power of 2 as n — oo, which therefore approaches 
oo. Therefore, n! cannot be bounded by a constant times c”. 


63. a) lim... aie = lim, 1/x =0 b) lim% zost = 
ime “* = lim,,., +; = 0 (using L'’Hôpital’s rule) 
i = à. 2x : 2 : 

c) Tim, 66 = lim o n2 = lims Tna = 0 (using 


L’H6pital’s rule) d) lim, ,,, ++ = lim, (1 += +4 +) = 


1#0 


65. 9? 


lim Xlogx _ 
x— o x2 
xlogx 
xlogx 
<2 
x 


67. No. Take f(x) = 1/2? and g(x) = 1/x. 69. a) Because 
lim, ..of()/g() = 0, [FQO|/lg@)| < 1 for sufficiently large 
x. Hence, |f(x)|_ < |g(x)| for x > k for some constant k. 
Therefore, f(x) is O(g(x)). b) Let f(x) = g(x) = x. Then 
fœ) is O(g(x)), but f(x) is not o(g(x)) because f(x)/g(x) = 1. 
71. Because f,(x) is o(g(x)), from Exercise 69(a) it follows 
that f(x) is O(g(x)). By Corollary 1, we have f,(x) + hx) 
is O(g(x)). 73. We can show that (n — i(i + 1) >n for 
i=0,1,...,2—1. Hence, (a)? = (n+ 1)((n— 1) - 2) - 
((n — 2) - 3) +++ (2-(n — 1) - (1 - n) È> n”. Therefore, 
2logn! > nlogn. 75. Compute that log 5! = 6.9 and 
(5 log 5)/4 ~ 2.9, so the inequality holds for n = 5. As- 
sume n > 6. Because n! is the product of all the integers from 
n down to 1, we have n! > n(n — 1)(n — 2) ++ [n/2] (be- 
cause at least the term 2 is missing). Note that there are more 
than n/2 terms in this product, and each term is at least as big 
as n/2. Therefore, the product is greater than (n/2)/?). Tak- 


ing the log of both sides of the inequality, we have log n! > 
n/2 

log (£) = 5 log 5 = (logn — 1) > (nlogn)/4, be- 

cause n > 4 implies logn — 1 > (logn)/2. 77. All are not 

asymptotic. 


Section 3.3 


1.00) 3.0(n7) 5.2n-1  7.Linear 9. O(n) 
11. a) procedure disjointpair(S,, Sz, ...,.S, : 
subsets of {1, 2, ...,}) 
answer := false 
for i := 1 ton 
for j:=i+1ton 
disjoint := true 
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fork :=1ton 
if k € S; and k € S; then disjoint := false 
if disjoint then answer := true 
return answer 

b) O(n?) 13. a) power := 1, y := 1; i := 1, power := 2, 
y := 3; i := 2, power := 4,y := 15 b) 2n multiplications 
and n additions 15. a) 2!” ~10°*! b) 10° c) 3.96 x 107 
d) 3.16 x 104 e)29 f)12 17.a)22"" p) 260-10" 
c) |2 V60-105 | œ% 2 x 10331768 d) 60,000,000 e) 7,745,966 
f)45 g)6 19.a)36 years b)13 days c) 19 minutes 
21. a) Less than 1 millisecond more b) 100 milliseconds 
more c) 2n+1 milliseconds more d) 3n? + 3n + 1 millisec- 
onds more e) Twice as much time f) 2°”+! times as many 
milliseconds g) n+ 1times as many milliseconds 23. The 
average number of comparisons is (3n+ 4)/2. 25. O(logn) 
27.O(n) 29. O(n?) 31. 0(n) 33.0(n) 335. O(log n) 
comparisons; O(n?) swaps 37. O(m(n—m)) 39. a) O(n”) 
b) O(n logn) 41. O(n?2") 43. a) doubles b) increases 
by 1 45. Use Algorithm 1, where A and B are now n xn 
upper triangular matrices, by replacing m by n in line 1, and 
having q iterate only from i to j, rather than from 1 to k. 
47. n(n+1)(n+2)/6 49. A(BC)D) 


Supplementary Exercises 


1. a) procedure last max(q,, ..., a, integers) 
max := a, 
last := 1 
i:=2 
while i < n 
if a; > max then 
max := d; 
last :=1 
i:=it+1 
return last 


b) 2n — 1 = O(n) comparisons 


3. a) procedure pair zeros(b,b, ...b,: bit string, n > 2) 
x:=b, 
y := b 
k:=2 
while k < n and (x #4 0 or y #0) 
k:=k+1 
x:=y 
y= by 
if x = 0 and y = 0 then print “YES” 
else print “NO” 


b) O(n) 


5. a) and b) 


procedure smallest and largest(a,, ay, ..., 
min := ay 


a, integers) 


max := a, 
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for i := 2 ton 

if a; < min then min := a; 

if a; > max then max := a; 
{min is the smallest integer among the input, and max is the 
largest} 


c)2n-2 


7. Before any comparisons are done, there is a possibility that 
each element could be the maximum and a possibility that 
it could be the minimum. This means that there are 2n dif- 
ferent possibilities, and 2n — 2 of them have to be elimi- 
nated through comparisons of elements, because we need to 
find the unique maximum and the unique minimum. We clas- 
sify comparisons of two elements as “virgin” or “nonvirgin,” 
depending on whether or not both elements being compared 
have been in any previous comparison. A virgin comparison 
eliminates the possibility that the larger one is the minimum 
and that the smaller one is the maximum; thus, each virgin 
comparison eliminates two possibilities, but it clearly cannot 
do more. A nonvirgin comparison must be between two ele- 
ments that are still in the running to be the maximum or two 
elements that are still in the running to be the minimum, and 
at least one of these elements must not be in the running for 
the other category. For example, we might be comparing x 
and y, where all we know is that x has been eliminated as the 
minimum. If we find that x > y in this case, then only one 
possibility has been ruled out—we now know that y is not the 
maximum. Thus, in the worst case, a nonvirgin comparison 
eliminates only one possibility. (The cases of other nonvirgin 
comparisons are similar.) Now there are at most [1/2] com- 
parisons of elements that have not been compared before, each 
removing two possibilities; they remove 2|n/2]| possibilities 
altogether. Therefore, we need 2n —2—2|n/2| more compar- 
isons that, as we have argued, can remove only one possibility 
each, in order to find the answers in the worst case, because 
2n — 2 possibilities have to be eliminated. This gives us a 
total of 2n — 2 — 2|n/2] + |n/2] comparisons in all. But 
2n — 2 — 2|n/2] + [n/2] = 2n — 2 — |n/2] =2n — 2 + 
[-n/2] =[2n—n/2] —2 = [3n/2]| —2, as desired. 9. The 
following algorithm has worst-case complexity O(n*). 
procedure equal sums(d,, az, ..., A, 
for i:=1ton 
for j :=i+ 1 ton {since we want i < j} 
for k:=1ton 
for /:=k+ 1 ton {since we want k < /} 
if a; +a; =a, + a, and (i j) + (k, D 
then output these pairs 
11. At end of first pass: 3, 1, 4, 5, 2, 6; at end of second 
pass: 1, 3, 2, 4, 5, 6; at end of third pass: 1, 2, 3, 4, 5, 6; 
fourth pass finds nothing to exchange and algorithm termi- 
nates 13. There are possibly as many as n passes through 
the list, and each pass uses O(n) comparisons. Thus, there are 
O(n?) comparisons in all. 15. Because logn < n, we have 
(nlogn+n?P < (n? +r’) < (2n?) = 8nf for all n > 0. This 
proves that (nlogn +n) is O(n), with witnesses C = 8 one 
k=0. 17.002") 19. Note that = 4-4*..3.5.55 


N 


s Lele: 7 21. All of these functions are of the 
same order. 23.2! 25. (log n)?, 2V!°2", n(log n)!!, 
n!®001 1.0001", n” 27. For example, f(n) = n?!"/2!*! and 
re = nln) 
29. a) procedure brute(a,, dy, ..., 
fori := 1 ton— 1 
forj:=i+1ton 
for k := 1 ton 
if a; + a; = a, then return true 
else return false 


d,, : integers) 


b) O(n’) 

31. For m,: w, and w3; for my: w, and w3; for m3: w, and 
w3; for w,;: m; and m; for wy: m; and m3; for w3: m, and 
m, 33. A matching in which each woman is assigned her 
valid partner ranking highest on her preference list is female 
optimal; a matching in which each man is assigned his valid 
partner ranking lowest on his preference list is male pessi- 
mal. 35. a) Modify the preamble to Exercise 64 in Sec- 
tion 3.1 so that there are s men my, m, ..., m, and t women 
Wi, Wo, ..., Wp A matching will contain min(s, t) marriages. 
The definition of “stable marriage” is the same, with the 
understanding that each person prefers any mate to being 
unmatched. b) Create |s—t| fictitious people (men or women, 
whichever is in shorter supply) so that the number of men 
and the number of women become the same, and put these 
fictitious people at the bottom of everyone’s preference lists. 
c) This follows immediately from Exercise 67 in Section 3.1. 
37.5;15 39. The first situation in Exercise37 41. a) For 
each subset S of {1, 2, ..., n}, compute Lies Ww. Keep track 
of the subset giving the largest such sum that 1 iš less than or 
equal to W, and return that subset as the output of the algo- 
rithm. b) The food pack and the portable stove 43. a) The 
makespan is always at least as large as the load on the pro- 
cessor assigned to do the lengthiest job, which must be at 
least max;-1,2,...n t; Therefore, the minimum makespan satis- 
fies this inequality. b) The total amount of time the processors 


need to spend working on the jobs (the total load) is mE Ép 


Therefore, the average load per processor is 1 ae The 
maximum load cannot be any smaller than the TEPA s the 
minimum makespan is always at least this large. 45. Pro- 
cessor 1: jobs 1, 4; processor 2: job 2; processor 3: jobs 3, 5 


CHAPTER 4 

Section 4.1 

1. a) Yes b)No c¢)Yes d)No 3. Suppose thata | b. 
Then there exists an integer k such that ka = b. Because 
a(ck) = bc it follows thata | bc. 5.Ifa | bandb | a, 
there are integers c and d such that b = ac anda = bd. 


Hence, a = acd. Because a # 0 it follows that cd = 1. Thus, 
either c = d = lorc = d = —l1. Hence, either a = b 
ora = —b. 7. Because ac | bc there is an integer k such 


that ack = bc. Hence, ak = b, soa | b. 9. It is given that 
b = m-a for some integer m. If a is not odd, then a = 2k 
for some integer k, whence b = 2km. This means by defini- 
tion that b is even. 11. If a is an integer that is not divisible 
by 3, then it must leave a remainder of either 1 or 2 when di- 
vided by 3. In the first case a = 3k + 1 for some integer k, so 
(a+ 1)(a+2) = (3k+2)(3k4+3) = 3(3k+2)(k+ 1) and so is di- 
visible by 3. In the second case a = 3k + 2 for some integer k, 
so (a+ 1)(a +2) = (3k +3)(3k +4) = 3(k + 1)(3k + 4) and so 
is divisible by 3. 13. a) 2,5 b)—11, 10 c) 34, 7 d)77, 0 
e) 0, Of) 0, 3g)-1, 2h)4,0 15. a) 7:00 b) 8:00 c) 10:00 
17. a) 10 b)8c)0d)9e)6f) 11 19. Ifd |a, then a = md 
for some integer m, and it follows that —a = (—m)d as well. 
Therefore, —(a div d) = —m and (—a) div d = —m, as de- 
sired. Conversely, if d does not divide a, then a = qd + r for 
some r with 0 < r < d. Here q = a div d. It follows that 

a = (—q)d—r = (—q— l)d + (d —r). Note that O < d -r < d, 
so by definition (—a) div d = —q — 1. But —(a div d) = —q, so 
(—a) div d #4 —(a div d). 21. If amod m = b mod m, then 
a and b have the same remainder when divided by m. Hence, 
a = qım +r and b = qam + r, where 0 < r < m. It follows 
that a — b = (qı — q2)m, so m | (a — b). It follows that a = b 
(mod m). 23. There is some b with (b — 1)k < n < bk. 
Hence, (b — 1) < n— 1 < bk. Divide by k to obtain 
b-—1 < n/k < bandb-1 < (n — 1)/k < b. Hence, 
[n/k]| = band |(n — 1)/k] = b- 1. 25.x mod m 
ifxmodm < [m/2] and (x mod m) — m if x mod m > 
[m/2|] 27. a)l b)2 c)3 d)9 29. a)l, 109 b) 40, 
89 c) —31, 222 d) —21,38259 31. a)—15 b)-7 c) 140 
33. —1, —26, —51, —76, 24, 49, 74,99 35. a)No b)No 
c) Yes d)No 37. a) 13 a)6 39. a)9 b)4 c)25 dO 
41. Let m = tn. Because a = b (mod m) there exists an integer 
s such that a = b + sm. Hence, a = b+(st)n, so a = b (mod n). 
43. a) Let m = c = 2, a = 0, and b = 1. Then 0 =ac=bc =2 
(mod 2), butO = a b = 1 (mod 2). b)Letm = 5, 
a=b = 3,c = 1, and d = 6. Then 3 = 3 (mod 5)and 1 = 6 
(mod 5), but 3! = 3 # 4 = 729 = 3° (mod 5). 45. By 
Exercise 44 the sum of two squares must be either 0 + 0 = 0, 
0+1 = 1,or1 +1 = 2, modulo 4, never 3, and therefore 
not of the form 4k + 3. 47. Because a = b (mod m), there 
exists an integer s such that a = b+ sm, soa -— b = sm. 
Then ak — bk = (a — bya"! + ab + + + ab + DI), 
k > 2, is also a multiple of m. It follows that a = b* (mod m). 
49. To prove closure, note that a -,, b = (a- b) mod m, which 
by definition is an element of Z,,,. Multiplication is associa- 
tive because (a -,, b) -,, canda -,, (b -n c) both equal 
(a - b- c) mod m and multiplication of integers is associa- 
tive. Similarly, multiplication in Z,,, is commutative because 
multiplication in Z is commutative, and 1 is the multiplicative 
identity for Z, because 1 is the multiplicative identity for Z. 
51. 0+,;0 = 0,0+51 = 1, 0+;2 = 2, 0+;3 = 3, 0+,;4 = 
4; 1+;1 = 2, 1+;2 = 3, 1+;3 = 4, 1+,;4 = 0;2+;2 = 
4, 2+53 = 0, 2+;4 = 1; 3+;3 = 1, 3+;4 = 2; 4+,4 = 3 and 
0-50 = 0,0-51 = 0, 0-;2 = 0,0-53 = 0,0-54 = 0; 1⁄1 = 
1, 1-52 = 2, 1-53 = 3, 1-54 = 452-52 = 4,2-53 = 1,244 = 


m 
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333-53 = 4,354 = 2;4,;4 = 1 53.f is onto but not 
one-to-one (unless d = 1); g is neither. 


Section 4.2 


1. a)1110 0111 b)i OOO! 1011 0100 œ©1 Olli 
11010110 1100 3.a)31 b)513 ©) 341 d)26,896 
5. a)l 0111 1010 b)11 1000 0100 œ)1 0001 0011 
d) 101 0000 1111 7.a)1000 0000 1110 b)1 001i 
0101 1010 1011 c) 10101011 1011 1010 d)1101 
1110 1111 1010 11001110 1101 9.1010 1011 1100 
1101 1110 1111 11. (B7B) 16 13. Adding up to 
three leading Os if necessary, write the binary expan- 
sion as (...b93b99by1b29b, 3012111003021 00)2-_ The 
value of this numeral is dog + 2bo, + 4b 9) + 8593 + 
2bio + 29D), + 26b + 27,3 + Bho + 22s, + 20b + 
2'b,, + +++, which we can rewrite as bog + 2b; + 4b9. + 
8ho3 + (bio + 2b), + 4by + 8b;3) < 24 + (bag + 205, + 4b + 
8b23) : 28 + ++. Now (bgbpbibio)h translates into the hex- 
adecimal digit h;. So our number is họ + hy - 24 + hy - 
28 + =- = ho +h, - 16 + h, - 16? + +, which is the 
hexadecimal expansion (...4,h,ho)g. 15. Adding up to 
two leading Os if necessary, write the binary expansion as 
(... bazb21ba0b12b11b10b02b01b00)2. The value of this numeral 
is bog + 2by + 4b + 2719 +24), + 25b; + 2° +2’) + 
28b», + ++, which we can rewrite as boo + 2b9, + 4bo + 
(bio + 2b), + 4b 49) + 23 + (bao + 2b; + 459) + 2° + ++. Now 
(babi bio) translates into the octal digit h;. So our number is 
hy thy > 2B +m -26+ = hy +h - 8th, - 8 +e, 
which is the octal expansion (...A,;h,ho)g. 17. 1 1101 1100 
1010 1101 0001, 1273), 19. Convert the given octal nu- 
meral to binary, then convert from binary to hexadecimal 
using Example 7. 21. a) 1011 1110, 10 0001 0000 0001 
b) 1 1010 1100, 1011 0000 0111 0011 c) 100 1001 1010, 
101 0010 1001 0110 0000 d) 110 0000 0000, 
10000000 0001 1111 1111 23. a) 1132, 144,305 b) 6273, 
2,134,272  c)2110, 1,107,667 d)57,777, 237,326,216 
25.436 27.27 29. The binary expansion of the integer 
is the unique such sum. 31. Let a = (a,_,d,_5 ... 4,dq)10: 


Thena = 10" 1a,_, + 10" 7a,-5 + = + 10a, + dp 
= anı + a2 + + + ay + a (mod 3), because 
10/ = 1 (mod 3) for all nonnegative integers j. It fol- 


lows that 3 | a if and only if 3 divides the sum of the dec- 
imal digits of a. 33. Let a= (a,_)4,_ ...d,dg).. Then 
a = dy + 2a, + 2a, + © +27 1a,_) = ay — a, +a 
a, + ++ + a,_, (mod 3). It follows that a is divisible by 
3 if and only if the sum of the binary digits in the even- 
numbered positions minus the sum of the binary digits in the 
odd-numbered positions is divisible by 3. 35. a) n is divis- 
ible by 4 if and only if the two rightmost digits of the decimal 
expansion, viewed as a two-digit integer (or the single digit 
if it’s a one-digit integer), is divisible by 4. b) n is divisible 
by 25 if and only if the two rightmost digits are 00 (or 0 if it’s 
a one-digit integer), 25, 50, or 75. c)n is divisible by 20 if 
and only if the rightmost digit of the decimal expansion is 0 
and the digit in the tens place (if it’s not a one-digit integer) 
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is 0, 2, 4, 6, or 8. 37. The base b representation of n has 
k digits iff b=! < n < be iff k —1 < logn < kiff 
[logn] +1 = k. 39.5(8" — 1)/7 41.a)-6 b)13 
c)—14 d)O 43. The one’s complement of the sum is found 
by adding the one’s complements of the two integers except 
that a carry in the leading bit is used as a carry to the last bit 
of the sum. 45. If m > 0, then the leading bit a,_, of the 
one’s complement expansion of m is 0 and the formula reads 
m= yie a,2'. This is correct because the right-hand side is 
the binary expansion of m. When m is negative, the leading 
bit a,_,; of the one’s complement expansion of m is 1. The 
remaining n — | bits can be obtained by subtracting —m from 
111... 1 (where there are n — 1 1s), because subtracting a bit 
from | is the same as complementing it. Hence, the bit string 
a„—2 «+ Ag is the binary expansion of (2”7! — 1) — (—m). Solv- 
ing the equation (2”-! —1)—(—m) = yo a;2' for m gives the 
desired equation because a„_; = 1. 47. a) —7 b) 13 c)-15 
d)—1 49. To obtain the two’s complement representation 
of the sum of two integers, add their two’s complement rep- 
resentations (as binary integers are added) and ignore any 
carry out of the leftmost column. However, the answer is 
invalid if an overflow has occurred. This happens when the 
leftmost digits in the two’s complement representation of the 
two terms agree and the leftmost digit of the answer differs. 
51. If m > 0, then the leading bit a,_, is O and the formula 
reads m = X2} a;2'. This is correct because the right-hand 
side is the binary expansion of m. If m < 0, its two’s comple- 
ment expansion has | as its leading bit and the remaining n— 1 
bits are the binary expansion of 2”-! — (—m). This means that 
(2"-!) — (—m) = ya a;2'. Solving for m gives the desired 
equation because a,_; = 1. 53. 4n 


55. procedure Cantor(x: positive integer) 
n= l f=] 
while (n+ 1) -f <x 
n:=n+1 
f:=fen 
yi=x 
while n > 0 
a, = |y/f] 
yi=y—a, -f 
f:=f/n 
n:i=n—1 
{x=a,n!+a,_\(n—D!+-++a,1)} 


57. First step: c = 0, d = 0, sọ = 1; second step: c =0,d = 1, 
sı = 0; third step: c = 1, d = 1, s, = 0; fourth step: c = 1, 
d = 1, s, = 0; fifth step: c = 1,d = 1, są = 1; sixth step: 
c=1,s;=1 
59. procedure subtract(a, b: positive integers, a > b, 
a = (a142 --- 49)95 
b = (b,—1bn2 --- by b9)2) 
B := 0 {B is the borrow} 
for] :=Oton-1 
if a; > b; + B then 


s; := a; — b; — B 


B:=0 


else 
Sj :=a;+2-b;-B 
B:= 1 


{(S,,-18)—2 --- S150) is the difference } 


61. procedure compare(a, b: positive integers, 


a = (aå, --»4,d9)9, b = (b bp-i -.. by b9)2) 
k:=n 
while a, = b, and k > 0 
k:=k-1 


if a, = b, then print “a equals b” 
if a, > b, then print “a is greater than b” 
if a, < b, then print “a is less than b” 


63. O(logn) 65. The only time-consuming part of the algo- 
rithm is the while loop, which is iterated q times. The work 
done inside is a subtraction of integers no bigger than a, which 
has log a bits. The result now follows from Example 9. 


Section 4.3 


1. 29,71,97 prime; 21,111,143 not prime 3. a) 2? - 11 
b)2-3?-7 ¢)3° d)7- 11-13 e) 11-101 f)2- 33. 
5-7-13-37 5.28.34.52.7 
7. procedure primetester(n : integer greater than 1) 
isprime := true 
d:=2 
while isprime and d < yn 
if n mod d = 0 then isprime := false 
else d:=d +1 
return isprime 


9. Write n = rs, wherer > 1l ands > 1. Then 
anal znal (2'ys-1 (=I HOF 7 +23 4 
-+ 1). The first factor is at least 27 — 1 = 3 and the second 
factor is at least 27 + 1 = 5. This provides a factoring of 
2” — 1 into two factors greater than 1, so 2” — 1 is composite. 
11. Suppose that log, 3 = a/b where a,b E€ Z* and b # 0. 
Then 2%/? = 3, so 24 = 3°. This violates the fundamental the- 
orem of arithmetic. Hence, log, 3 is irrational. 13. 3,5, and 
7 are primes of the desired form. 15. 1,7, 11, 13, 17, 19, 23, 
29 17.a) Yes b)No c) Yes d) Yes 19. Suppose that n 
is not prime, so that n = ab, where a and b are integers greater 
than 1. Because a > 1, by the identity in the hint, 2“ — 1 is a 
factor of 2” — 1 that is greater than 1, and the second factor in 
this identity is also greater than 1. Hence, 2” — 1 is not prime. 
21.a)2 b)4 ce) 12) 23. d(p*)=p*k—p*! 25.a) 39-53 
b) 1 c) 231 d) 41-43-53 e) 1 f) 1111 27. a) 2!!-37.5°.73 
b) 2? -37.55.77 - 11-13-17 ©) 237! d)41 - 43-53 
e) 2!23135!777! f) Undefined 29. gcd (92928, 123552) = 
1056; Icm(92928, 123552) = 10,872,576; both products 
are 11,481,440,256. 31. Because min(x, y) + max(x, y) = 
x + y, the exponent of p; in the prime factorization of 
gcd(a, b) - lcm(a, b) is the sum of the exponents of p; in the 
prime factorizations of a and b. 33.a)6 b)3 oc) 11 d)3 
e)40 f)12 35. 9 37. By Exercise 36 it follows that 
gcd(2? — 1, (24 — 1) mod (2? — 1)) = ged(2° — 1, 27mod> — 1), 


Because the exponents involved in the calculation are b and 
a mod b, the same as the quantities involved in computing 
gcd(a, b), the steps used by the Euclidean algorithm to com- 
pute gcd(2¢—1, 2? —1) run in parallel to those used to compute 
gcd(a, b) and show that gcd(2% — 1, 2? — 1) = 28°44) — 1, 
39. a)1 = (-1)-104+1-11 b)1l = 21-214 (—10) 44 
c)12 = (-1)-36+ 48 djl = 13-55 + (-21) - 34 
e) 3 = 11-213+(—20)-117 f) 223 = 1-0+1-223 g) 1 =37- 
2347 + (—706)- 123 h) 2= 1128 - 3454 +(—835) -4666 i) 1 = 
2468 - 9999 + (—2221)- 11111 41. (—3) -26 +1 -91 = 13 
43. 34-144 + (—55)- 89 = 1 


45. procedure extended Euclidean(a, b: positive integers) 
xi=a 
y:=b 
oldolds := 1 
olds :=0 
oldoldt := 0 
oldt := 1 
while y 4 0 
q := x div y 
r := x mod y 
xi=y 
yi=sry 
s := oldolds — q - olds 
t := oldoldt — q - oldt 
oldolds := olds 
oldoldt := oldt 
olds := s 
oldt := t 
{gcd(a, b) is x, and (oldolds)a + (oldoldt)b = x} 


47. a) a, = 1 if n is prime and a, = 0 otherwise. b) a, is the 
smallest prime factor of n with a; = 1. c) a, is the number of 
positive divisors of n. d) a, = 1 if n has no divisors that are 
perfect squares greater than | and a, = 0 otherwise. e)a, is 
the largest prime less than or equal to n. f) a, is the product 
of the first n — 1 primes. 49. Because every second integer 
is divisible by 2, the product is divisible by 2. Because every 
third integer is divisible by 3, the product is divisible by 3. 
Therefore, the product has both 2 and 3 in its prime factoriza- 
tion and is therefore divisible by3-2=6. 51.n = 1601 is 
a counterexample. 53. Setting k = a + b + 1 will produce 
the composite number a(a + b + 1) + b = @œ +ab +a +b = 
(a+1)(a+b). 55. Suppose that there are only finitely many 
primes of the form 4k+3, namely, q4, q2, ..., qn, where q; = 3, 
q = 7, and so on. Let Q = 4q,q, ++: q, — 1. Note that Q is of 
the form 4k +3 (where k = qiq + q, — 1). If Q is prime, then 
we have found a prime of the desired form different from all 
those listed. If Q is not prime, then Q has at least one prime 
factor notin the list q4, q2, ..., g,, because the remainder when 
Q is divided by qj is qj - 1, and qj - 1 Æ 0. Because all odd 
primes are either of the form 4k + 1 or of the form 4k + 3, and 
the product of primes of the form 4k + 1 is also of this form 
(because (4k + 1)(4m + 1) = 4(4km +k + m) + 1), there must 
be a factor of Q of the form 4k + 3 different from the primes 
we listed. 57. Given a positive integer x, we show that there 
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is exactly one positive rational number m/n (in lowest terms) 
such that K(m/n) = x. From the prime factorization of x, read 
off the m and n such that K(m/n) = x. The primes that occur 
to even powers are the primes that occur in the prime factor- 
ization of m, with the exponents being half the corresponding 
exponents in x; and the primes that occur to odd powers are 
the primes that occur in the prime factorization of n, with the 
exponents being half of one more than the exponents in x. 


Section 4.4 


1.15.7 = 105 = 1 (mod 26) 3.7 5. a)7 b)52 c) 34 
d)73 7. Suppose that b and c are both inverses of a mod- 
ulo m. Then ba = 1 (mod m) and ca = 1 (mod m). Hence, 
ba = ca (mod m). Because gcd(a, m) = 1 it follows by The- 
orem 7 in Section 4.3 that b = c (mod m). 9.8 11. a) 67 
b)88 c)146 13.3and6 15.Letm’ = m/ gcd(c, m). 
Because all the common factors of m and c are divided out of 
m to obtain m’, it follows that m’ and c are relatively prime. 
Because m divides ac — bc = (a — b)c, it follows that m’ 
divides (a — b)c. By Lemma 2 in Section 4.3, we see that 
m divides a — b, soa = b(mod m’). 17. Suppose that 
xX? = 1 (mod p). Then p divides x? — 1 = (x + 1)(x — 1). By 
Lemma 3 in Section 4.3 it follows that p | x+ 1 orp |x- 1, 
so x = —1 (mod p) orx = 1(mod p). 19. a) Suppose that 
ia = ja (mod p), where 1 < i < j < p. Then p divides 
ja — ia = a(j — i). By Lemma 3 in Section 4.3, because a is 
not divisible by p, p divides j — i, which is impossible because 
j—iis a positive integer less than p. b) By part (a), because no 
two of a, 2a, ..., (p — l)a are congruent modulo p, each must 
be congruent to a different number from 1 to p — 1. It follows 
thata-2a-3a--+-(p-—1)-a21-2-3-+--@M-1) 
(mod p). It follows that (p —1)!-a?-! = p—1 (mod p). ¢) By 
Wilson’s theorem and part (b), if p does not divide a, it fol- 
lows that (—1)-a?~! = —1 (mod p). Hence, a?~! = 1 (mod p). 
d) If p | a, then p | a’. Hence, a? = a= 0 (mod p). If p does 
not divide a, then a?~! =a (mod p), by part (c). Multiplying 
both sides of this congruence by a gives a? = a (mod p). 
21. All integers of the form 323 + 330k, where k is an integer 
23. All integers of the form 53 + 60k, where k is an integer 
25. procedure chinese(m,, m, ...,m,, : relatively 
prime positive integers ; a, av, ... , a, : integers) 
m:=1 
fork := 1 ton 
m:i=m-m, 
for k := 1 ton 
M, := m/m; 
Y = M7! mod m; 
x:=0 
fork := 1 ton 
x= x+a,M,y, 
while x > m 
x:=x-m 
return x {the smallest solution to the system 
{x =a, (mod m,),k = 1,2,...,n }} 
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27. All integers of the form 16 + 252k, where k is an integer 
29. Suppose that p is a prime appearing in the prime factor- 
ization of m,m, «+: m,. Because the m;s are relatively prime, p 
is a factor of exactly one of the m;s, say m;. Because m, divides 
a—b, it follows that a—b has the factor p in its prime factoriza- 
tion to a power at least as large as the power to which it appears 
in the prime factorization of m,. It follows that m,m, ++ m, di- 
videsa—b,soa=b (mod mm, ---m,).  31.x=1 (mod 6) 
33.7 35.a?-a=a-a?~ =a?-!=1 (mod p) 37. a) By 
Fermat’s little theorem, we have 2!? = 1 (mod 11). Hence, 
2340 = (2!)34 = 134 = 1 (mod 11). b) Because 32 = 1 
(mod 31), it follows that 24° = (25) = 326 = 16 = 1 
(mod 31). c) Because 11 and 31 are relatively prime, and 
11 - 31 = 341, it follows by parts (a) and (b) and Exer- 
cise 29 that 2749 = 1 (mod 341). 39. a)3, 4,8 b) 983 
41. Suppose that q is an odd prime with q | 2? — 1. By Fer- 
mat’s little theorem, q | 217! — 1. From Exercise 37 in Section 
4.3, gcd(2? — 1, 217! — 1) = 28°U?-4-) _ 1, Because q is a com- 
mon divisor of 2? — 1 and 247! — 1, gcd(2? — 1,29! — 1) > 1. 
Hence, gcd(p, q — 1) = p, because the only other possibility, 
namely, gcd(p, q — 1) = 1, gives us ged(2? — 1,247! — 1) = 1. 
Hence, p | q — 1, and therefore there is a positive integer 
m such that q — 1 = mp. Because q is odd, m must be 
even, say, m = 2k, and so every prime divisor of 2? — 1 
is of the form 2kp + 1. Furthermore, the product of num- 
bers of this form is also of this form. Therefore, all divisors 
of 2? — 1 are of this form. 43. Mj, is not prime; M,; is 
prime. 45. First, 2047 = 23 - 89 is composite. Write 
2047 — 1 = 2046 = 2-1023, so s = 1 and ¢ = 1023 in the defi- 
nition. Then 2!073 = (2!!)3 = 204893 = 1%=1 (mod 2047), 
as desired. 47. We must show that b”? = 1 (mod 2821) 
for all b relatively prime to 2821. Note that 2821 = 7-13-31, 
and if gcd(b, 2821) = 1, then gcd(b, 7) = gced(b, 13) = 
gcd(b, 31) = 1. Using Fermat’s little theorem we find that 
b = 1 (mod 7), b? = 1 (mod 13), and b’? = 1 (mod 31). 
It follows that b?829 = (b°)*7° = 1 (mod 7), b” = (b!?)? = 
1 (mod 13), and 57° = (439) = 1 (mod 31). By Ex- 
ercise 29 (or the Chinese remainder theorem) it follows that 
b°820 = 1 (mod 2821), as desired. 49. a) If we multiply 
out this expression, we get n = 1296m? + 396m? + 36m + 1. 
Clearly 6m | n — 1, 12m|n — 1, and 18m |n — 1. There- 
fore, the conditions of Exercise 48 are met, and we conclude 
that n is a Carmichael number. b) Letting m = 51 gives 
n = 172,947,529. 51.0 = (0,0), 1 = (0, 1,2 = (2, 2), 
3 = (0,3),4 = (4),5 = (2,0),6 = (0, 1),7 = (1, 2), 
8 = (2,3),9 = (0,4), 10 = (1,0), 11 = (2, 1), 12 = (0, 2), 
13 = (1, 3), 14 = (2,4) 53. We have m, = 99, m, = 98, 
m, = 97, and m, = 95, so m = 99 - 98 - 97 - 95 = 89,403,930. 
We find that M, = m/m = 903,070, M, = m/m, = 912,285, 
M, = m/m, = 921,690, and M, = m/m, = 941,094. 
Using the Euclidean algorithm, we compute that y) = 37, 
y2 = 33, y; = 24, and y, = 4 are inverses of M, modulo m, 
for k = 1, 2, 3, 4, respectively. It follows that the solution is 
65 - 903,070 - 37 + 2 - 912,285 - 33 + 51 -921,690 - 24+ 10 - 
941,094 -4 = 3,397,886,480 = 537,140 (mod 89,403,930). 
55. log, 5 = 16, log, 6 = 14 57. log; 1 = 0, log; 2 = 14, 
log, 3 = 1, log,4 = 12, log,5 = 5,1log,6 = 15, 


log; 7 = 11, log,8 = 10, log,;9 = 2, log, 10 = 3, 
log; 11 = 7, log, 12 = 13, log, 13 = 4, log, 14 = 9, 
log; 15 = 6, log; 16 = 8 59. Assume that s is a solu- 
tion of x? = a (mod p). Then because (—s)* = s? 
also a solution. Furthermore, s # —s (mod p). Otherwise, 
p | 2s, which implies that p | s, and this implies, using 
the original assumption, that p | a, which is a contradiction. 
Furthermore, if s and ¢ are incongruent solutions modulo p, 
then because s$ = f° (mod p), p | s$? — £. This implies that 
p | (s +t)(s — t), and by Lemma 3 in Section 4.3, p | s — t 
orp | s+t,sos = t(modp) ors = —t (mod p). Hence, 
there are at most two solutions. 61. The value of (=) de- 
pends only on whether a is a quadratic residue modulo p, that 
is, whether x? = a (mod p) has a solution. Because this de- 
pends only on the equivalence class of a modulo p, it follows 
that (5) = (C) ifa = b (mod p). 63. By Exercise 62, 
(£) (5) = g?-V/2p0-D/2 = (abyP-V/2 = (*) (mod p). 
65. x= 8, 13, 22, or 27 (mod 35) 67. Compute r° mod p for 
e = 0,1,2,..., p — 2 until we get the answer a. Worst case 
and average case time complexity are O(p log p). 


, —s is 


Section 4.5 


1.91,57,21,5  3.a)7,19,7,7, 18,0 b) Take the next avail- 
able space mod 31. 5.1,5,4,1,5,4,1,5,4,.... 7.2, 6,7, 
10, 8, 2, 6, 7, 10, 8, ... 9. 2357, 5554, 8469, 7239, 4031, 
2489, 1951, 8064 11.2, 1,1, 1, ... 13. Only string (d) 
15.4 17. Correctly, of course 19. a) Not valid b) Valid 
c) Valid d)Not valid 21.a)No b)5 oc)7 d)8 
23. Transposition errors involving the last digit 25. a) Yes 
b) No c) Yes d)No 27. Transposition errors will be de- 
tected if and only if the transposed digits are an odd num- 
ber of positions apart and do not differ by 5. 29. a) Valid 
b) Not valid c) Valid d) Valid 31. Yes, as long as the 
two digits do not differ by 7 33. a) Not valid b) Valid 
c) Valid d) Not valid 35. The given congruence is equiva- 
lent to 3d, + 4d, + 5d; + 6d, + 7d; + 8d + 9d, + 10d, = 0 
(mod 11). Transposing adjacent digits x and y (with x on the 
left) causes the left-hand side to increase by x — y. Because 
x Æ y (mod 11), the congruence will no longer hold. There- 
fore, errors of this type are always detected. 


Section 4.6 


1.a) GR QRW SDVV JR b) QB ABG CNFFTB c) QX UXM 
AHJJ ZX 3.a) KOHQV MCIF GHSD b) RVBXP TJPZ 
NBZX c)DBYNE PHRM FYZA _ 5. a) SURRENDER 
NOW b)BE MY FRIEND c) TIME FOR FUN 7. TO 
SLEEP PERCHANCE TO DREAM 9.ANY SUFFI- 
CIENTLY ADVANCED TECHNOLOGY IS INDISTIN- 
GUISHABLE FROM MAGIC Il.p = 7c + 13 mod 26 
13.a = 18,b = 5 15. BEWARE OF MARTIANS 
17. Presumably something like an affine cipher 
19. HURRICANE 21.The length of the key may well 


be the greatest common divisor of the distances between 
the starts of the repeated string (or a factor of the gcd). 
23. Suppose we know both n = pq and (p — 1)(q — 1). To find 
p and q, first note that (p — 1)(q -l)=pq-p-qtl= 
n — (p + q) + 1. From this we can find s = p + q. Because 
q = s — p, we have n = p(s — p). Hence, p? — ps +n = 0. We 
now can use the quadratic formula to find p. Once we have 
found p, we can find q because q = n/p. 25. 2545 2757 
1211 27. SILVER 29. Alice sends 58 mod 23 = 16 to 
Bob. Bob sends 5° mod 23 = 20 to Alice. Alice computes 
20° mod 23 = 6 and Bob computes 16° mod 23 = 6. 
The shared key is 6. 31. 2186 2087 1279 1251 0326 0816 
1948 33. Alice can decrypt the first part of Cathy’s mes- 
sage to learn the key, and Bob can decrypt the second part 
of Cathy’s message, which Alice forwarded to him, to learn 
the key. No one else besides Cathy can learn the key, be- 
cause all of these communications use secure private keys. 
35. Working modulo n?, E(m, + m) = g™*™(ry7)" = 
(g™ ry (g"? 15) = En) - Em). 


Supplementary Exercises 


1. The actual number of miles driven is 46,518 + 100,000k 
for some natural number k. 3. 5,22, —12,—29 5. Because 
ac = bc (mod m), there is an integer k such that ac = bc + 
km. Hence, a — b = km/c. Because a — b is an integer, 
c | km. Letting d = gcd(m, c), write c = de. Because no 
factor of e divides m/d, it follows that d | m and e | k. Thus, 
a—b = (k/e)(m/d), where k/e E€ Zand m/d € Z. There- 
fore,a = b (mod m/d). 7. Proof of the contrapositive: 
If n is odd, then n = 2k + 1 for some integer k. Therefore, 
n? +1 = (2k + 1? +1 = 4k 4+4k +2 = 2 (mod 4). But 
perfect squares of even numbers are congruent to 0 modulo 4 
(because (2m)? = 4m7), and perfect squares of odd numbers 
are congruent to 1 or 3 modulo 4, so n? + 1 is not a perfect 
square. 9. nis divisible by 8 if and only if the binary expan- 
sion of n ends with 000. 11. We assume that someone has 
chosen a positive integer less than 2”, which we are to guess. 
We ask the person to write the number in binary, using leading 
Os if necessary to make it n bits long. We then ask, “Is the first 
bit a 1?”, “Is the second bit a 1?”, “Is the third bit a 1?”, and so 
on. After we know the answers to these n questions, we will 
know the number, because we will know its binary expansion. 
13. (apa, --- d1ao)io = Èro 10a, = Proa, (mod 9) 
because 10 = 1 (mod 9) for every nonnegative integer k. 
15. Because for all k < n, when Q, is divided by k the re- 
mainder will be 1, it follows that no prime number less than 
or equal to n is a factor of Q. Thus, by the fundamental theo- 
rem of arithmetic, Q,, must have a prime factor greater than n. 
17. Take a = 10 and b= 1 inDirichlet’s theorem. 19. Every 
number greater than 11 can be written as either 8+2n or 9+2n 
forsomen>2. 21. Assume that every even integer greater 
than 2 is the sum of two primes, and let n be an integer greater 
than 5. If n is odd, write n = 3 + (n — 3) and decompose 
n—3 = p + q into the sum of two primes; if n is even, then 
write n = 2 + (n — 2) and decompose n — 2 = p + q into 
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the sum of two primes. For the converse, assume that every 
integer greater than 5 is the sum of three primes, and let n be 
an even integer greater than 2. Write n +2 as the sum of three 
primes, one of which is necessarily 2, so n +2 =2+p+q, 
whence n = p+ q. 23. Recall that a nonconstant poly- 
nomial can take on the same value only a finite number of 
times. Thus, f can take on the values 0 and +1 only finitely 
many times, so if there is not some y such that f(y) is com- 
posite, then there must be some x, such that +f (xọ) is prime, 
say p. Look at f(xọ + kp). When we plug x9 + kp in for x in 
the polynomial and multiply it out, every term will contain 
a factor of p except for the terms that form f (xo). Therefore, 
So + kp) = f(x) + mp = (m + 1)p for some integer m. As k 
varies, this value can be 0, p, or —p only finitely many times; 
therefore, it must be a composite number for some values of k. 
25.1 27.1 29. If not, then suppose that g,, q2, ...,¢,, are 
all the primes of the form 6k + 5. Let Q = 6q,q) = qn — 1. 
Note that Q is of the form 6k + 5, where k = qiqa qa — 1. 
Let Q = piP + p, be the prime factorization of Q. No p; 
is 2, 3, or any qj» because the remainder when Q is divided 
by 2 is 1, by 3 is 2, and by q; is q; — 1. All odd primes other 
than 3 are of the form 6k + 1 or 6k + 5, and the product of 
primes of the form 6k + 1 is also of this form. Therefore, at 
least one of the p;s must be of the form 6k + 5, a contradic- 
tion. 31. The product of numbers of the form 4k + 1 is of 
the form 4k + 1, but numbers of this form might have num- 
bers not of this form as their only prime factors. For exam- 
ple, 49 = 4. 12 + 1, but the prime factorization of 49 is 
7-7=(4-1+3)(4-1+4+3). 33. a) Not mutually relatively 
prime b) Mutually relatively prime c) Mutually relatively 
prime d) Mutually relatively prime 35.1 37.x = 28 

(mod 30) 39. By the Chinese remainder theorem, it suffices 
to show that n? — n = 0 (mod 2), n? — n = 0 (mod 3), 
and n? — n = 0 (mod 5). Each in turn follows from apply- 
ing Fermat’s little theorem. 41. By Fermat’s little theorem, 
pt! = 1 (mod q) and clearly q7! = 0 (mod q). Therefore, 
pt! +q! =1+0=1 (mod q). Similarly, p! + q’-! = 1 
(mod p). It follows from the Chinese remainder theorem that 
pt! + q-! = 1 (mod pq). 43. If a; is changed from x 
to y, then the change in the left-hand side of the congruence 
is either y — x or 3(y — x), modulo 10, neither of which 
can be 0 because 1 and 3 are relatively prime to 10. There- 
fore, the sum can no longer be 0 modulo 10. 45. Working 
modulo 10, solve for dọ. The check digit for 11100002 is 5. 
47. PLEASE SEND MONEY 49. a) QAL HUVEM AT 
WVESGB b) QXB EVZZL ZEVZZRFS 


CHAPTER 5 


Section 5.1 


1. Let P(n) be the statement that the train stops at station n. 
Basis step: We are told that P(1) is true. Inductive step: We 
are told that P(n) implies P(n + 1) for each n > 1. Therefore, 
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by the principle of mathematical induction, P(n) is true for all 
positive integersn. 3. a) 1? = 1-2-3/6 b) Both sides of P(1) 
shown in part (a) equal 1. ¢) 17+27+++-+k? = k(k+1)(2k+ 
1)/6 d) For each k > 1 that P(k) implies P(k + 1); in other 
words, that assuming the inductive hypothesis [see part (c)] 
we can show 174+2?+4-+-+4?+(k+1)? = (k+1)(k+2)(2k+3)/6 
o (1? +2 + th) + (kt 1% = [kk + 12k + 
1)/6] + (k + IP =[Kk + 1)/O)[kKQk + 1) + 6(k+ 
LD] = [(k + 1)/6(2k? + 7k + 6) = [(k + 1)/6](k + 
2)(2k + 3) = (k + 1)(k + 2)(2k + 3)/6 £) We have completed 
both the basis step and the inductive step, so by the princi- 
ple of mathematical induction, the statement is true for every 
positive integern. 5. Let P(n) be “1? +3? +- +(2n+ 1} = 
(n + 1)(2n + 1)(2n + 3)/3.” Basis step: P(O) is true because 
1? =1=(0+1)(2-041)(2-04+3)/3. Inductive step: Assume 
that P(k) is true. Then 17+37+4---+(2k+1)?+[2(k+ D417 = 
(k+ 1)(2k + 1)(2k + 3)/3 + (2k + 3)? = (2k + 3)[(k + 1)(2k + 
1)/3 + (2k + 3)] = (2k +3)(2k? + 9k + 10)/3 = (2k + 3)\(2k + 
5)(k + 2)/3 = [(k + 1) + 1][2(k + 1) + 1J[2(k + 1) + 3)/3. 
7. Let P(n) be se 3-5) = 3(5"t! — 1)/4.” Basis step: 
P(O) is true because E 3-5) = 3 = 3(5! — 1)/4. In- 
ductive step: Assume that DA 3-5 = 3(5*! — 1)/4. Then 
yo 35 =O a6 asi = 
T +4. 5l L 1)/4 = 3654? — 1)/4. 9.a)2+4+ 
6 +-+ 2n=n(n + 1) b) Basis step: 2 = 1 - (1+ 1) is true. 
Inductive step: Assume that 2 + 4 + 6 +- + 2k = k(k + 1). 
Then (2 +4 +6+--+2k)+2(k+1)=k(k+1)+2(k+1)= 
(k + 1)(kK + 2). 11.a) dja 1/2 = (2" — 1)/2” b) Basis 


step: P(1) is true because + > = (2!— 1)/2!. Inductive step: 
k+l 
Assume that X 1/2 = * Qk — 1)/2*. Then 2a > 
k 1l 1 2-1 1 221 2i 
(Qiz JT Je = 7k F k+l = a = k+l * 13. Let 


P(n) be “17-2743? = ++ +(-1)" in? = (-1)""! n(n +1) /2.” Ba- 
sis step: P(1) is true because 1° = 1 = (—1)°1*. Inductive step: 
Assume that P(k) is true. Then 1? — 27 +3? — «+» +(—1)& 1k? + 
(-1(kK + 1} = (DEK + 1)/2 + (—1}(k + 1} = 
(CDEC + DI—k/2 + (k + 1] = (CD(k + DIG/2) + 1] = 
(1) (k+ Ik +2)/2. 15. Let P(n) be “1-24+2-3 4-04 
n(n+ 1) = n(n+ 1)(n4+2)/3.” Basis step: P(1) is true because 
1-2=2= 104) + 2)/3. Inductive step: Assume that 
P(k) is true. Then 1-24+2-34+--+k(kK+1)+(k+ 1(kK+2) = 
[k(k+ 1)(k+2)/3]+(k+1)\(k+2) = (k+ 1)(K+2)[(K/3)4+ 1] = 
(k + 1)(k + 2)(k + 3)/3. 17. Let P(n) be the statement that 
144244344. tnt =n(n+1)(2n+1)3n? +3n—1)/30. PCL) 
is true because 1 - 2-3 - 5/30 = 1. Assume that P(k) is true. 
Then (144244344 +--+) 4+(k+ 1) = k(k+ 1)(2k+ 1K? + 
3k — 1)/30 + (k + 1)* = [(k + 1)/30][K(2k + 1)\(3k? +3k— 1) + 
30(K+1)3] = [(k+1)/30](6kf +39k3 +91k?+89k +30) = [(k+ 
1)/30](k+2)(2k+3)[3(k+1)?+3(k+1)—1]. This demonstrates 
that P(k+ 1)is true. 19.a) 1+ > <2- + b) This is true be- 
cause 5/4is less than 6/4. c) 1+ tett < 2-2 d) For each 
k > 2 that P(k) implies P(A + 1); in other words, we want to 


show that assuming the inductiyè hypothesis [see part (c)] we 


can show 1 + 4 + ~ oe acre 2- = Rares -+ 
He jee le <2-1+ AN r 1 Je kK?+2k+1—k 
D? Tr k (k+l? k(k+1)? 
=k a i =2-+-—_ <2-— f) Wehave 
Kk+l2 K+ k+l ket? k+1 


completed both the basis step and the inductive step, so by the 
principle of mathematical induction, the statement is true for 
every integer n greater than 1. 21. Let P(n) be “2” > n?” 
Basis step: P(5) is true because 25 = 32 > 25 = 5°. In- 
ductive step: Assume that P(k) is true, that is, 2% > k?. Then 
12.2% > R+ > RP 44k >P4+2k4+1=(k4+ 17% 
becausek>4. 23. By inspection we find that the inequality 
2n + 3 < 2” does not hold for n = 0, 1, 2, 3. Let P(n) be the 
proposition that this inequality holds for the positive integer n. 
P(4), the basis case, is true because 2 - 4 + 3 = 11 < 16 = 24. 
For the inductive step assume that P(k) is true. Then, by the 
inductive hypothesis, 2(k + 1) +3 = (2k +3) +2 < 2% +2. 
But because k > 1, 2% +2 < 2‘ + 2* = 2%! This shows that 
P(k+1)istrue. 25. Let P(n) be “1+nh < (1+), h > —1.” 
Basis step: P(0) is true because 1 +0- h =1 <1 = (1 +h. 
Inductive step: Assume 1 + kh < (1 + h)*. Then because 
d+h>004+h!=(14+hH04+h)* (A+A + kh) = 
1+(k+1)h+ kk > 1+(k+1)h. 27.Let P(n) be 


“1/ V1 +1/V2+1/V3 + +1/yn>2 (vn = 1): 
Basis step: P(1) is true because 1 > 2 (v2- 1). Induc- 
tive step: Assume that P(k) is true. Then 1 + 1/2 + = + 


1/Vk + 1/Vk+1 > 2(vk+1-1) + 1/\V/k + 1. If we 
show that 2 k+1-1) +1/ye+1 > 2/ 


it follows that P(k + 1) is true. This inequality is equiv- 


alent to 2 (Vie#2- Vk+ 1) < 1/Vk+1, which is 
equivalent to 2 ( Yk+2- yk +1) (Vk+2+ Vk+1) < 


VkK+1/Vk+1 + VkK4+2/Vk4+1. This is equivalent to 
2 < 1+ VWk+2/Vk+1, which is clearly true. 29. Let 


P(n) be “Hy, < 1 + n?” Basis step: P(O) is true because 


Hy = H = 1 < 1 + O. Inductive step: as 
k+1 

that Ay: < 1 + k. Then Agus = H+ > 2k41 3 < 

1+k+ž (4) <1+k+1=1+(k+1). 31. Basis 


step: 1? + 1 = 2 is divisible by 2. Inductive step: Assume 
the inductive hypothesis, that k? + k is divisible by 2. Then 
(k+1}+(k+1) =k 4+2k414k41= (724k) 4+2K+1), 
the sum of a multiple of 2 (by the inductive hypothesis) and a 
multiple of 2 (by definition), hence, divisible by 2. 33. Let 
P(n) be “n° — n is divisible by 5.” Basis step: P(O) is true 
because 0° — 0 = 0 is divisible by 5. Inductive step: As- 
sume that P(k) is true, that is, k? — 5 is divisible by 5. Then 
(k+1P —(k +1) = (k5 +:5k4 + 10K +10k?+5k+1)-(k+1) = 
(K —k) +5(k4 +.2k3 + 2k? + k) is also divisible by 5, because 
both terms in this sum are divisible by 5. 35. Let P(n) be 
the proposition that (2n — 1)? — 1 is divisible by 8. The basis 
case P(1) is true because 8 | 0. Now assume that P(k) is true. 
Because [(2(k + 1) — 1 — 1 = [(2k — 1)? — 1] + 8k, P(k + 1) 
is true because both terms on the right-hand side are divisible 


by 8. This shows that P(n) is true for all positive integers n, so 
m? — 1 is divisible by 8 whenever m is an odd positive integer. 
37. Basis step: 111+! + 127171 = 121 + 12 = 133 Inductive 
step: Assume the inductive hypothesis, that 11+! + 122”-! is 
divisible by 133. Then 110+D+ 4 1220+D-1 = 11 . 11h + 
144 - 1277! = 41-11! + (11 + 133). 12%! = 
11(11"*! + 127-1!) + 133 - 12?”-1, The expression in paren- 
theses is divisible by 133 by the inductive hypothesis, and 
obviously the second term is divisible by 133, so the en- 
tire quantity is divisible by 133, as desired. 39. Basis step: 
A, CB, tautologically implies that N4 E Nai B,. Induc- 
tive step: Assume the inductive hypothesis that if A, T B; for 
j= ly? .. .,k, then N% A; c NÉ, B, Wë want to show that 
if A; GRE 3, „k+l, then ŽIA; c NE B, 

Let x be an a elem of aan Aj = TE Aj) A Aga: 

Because x € nh, , we know by the inductive hypothe- 


sis that x € ae eae x E A we know from 
the given fact that rs C Bı thatx € B,,,. There- 
fore, x € (Ak B,) N Bai = MELB, 41. Let P(n) be 
“(A UA, Us UA DNB = (A, N B) U (A) A B)U =- U (A, N 
B).” Basis step: P(1) is trivially true. Inductive step: Assume 
that P(k) is true. Then (Aj UA, U + UA, U Au) NB = 
[(A UA, U +++ VA,) UA; ] NB = [(A, UA U = UA;,) NB] U 
(A,,,NB) = [(A;NB)U(A,NB) U-U(A, NB)] U (Ay, N B) = 
(A; N B) U (A, N B) Us U (A; N B) U (Ay, NA B). 
43. Let P(n) be “Ui Ak = Ni A,.” Basis step: P(1) is 
trivially true. Inductive step: Assume that P(k) is true. Then 
Uae A; = (Ui: 4) VA = (Ui 4) NA) = 
(Ni A) N Apl = A A;. 45. Let P(n) be the state- 
ment that a set with n dana has n(n — 1)/2 two-element 
subsets. P(2), the basis case, is true, because a set with two 
elements has one subset with two elements—namely, itself— 
and 2(2 — 1)/2 = 1. Now assume that P(k) is true. Let 
S be a set with k + 1 elements. Choose an element a in S 
and let T = S — {a}. A two-element subset of S either 
contains a or does not. Those subsets not containing a are 
the subsets of T with two elements; by the inductive hypoth- 
esis there are k(k — 1)/2 of these. There are k subsets of 
S with two elements that contain a, because such a subset 
contains a and one of the k elements in 7. Hence, there are 
k(k — 1)/2 +k = (k + 1)k/2 two-element subsets of S. This 
completes the inductive proof. 47. Reorder the locations if 
necessary so that x, < x) < x3 < + < x4. Place the first 
tower at position t, = x, + 1. Assume tower k has been placed 
at position tį. Then place tower k + 1 at position t}; =x+1, 
where x is the smallest x; greater thant, + 1. 49. The two 
sets do not overlap if n + 1 = 2. In fact, the conditional state- 
ment P(1) > P(2) is false. 51. The mistake is in applying 
the inductive hypothesis to look at max(x — 1, y — 1), because 
even though x and y are positive integers, x — 1 and y — 1 
need not be (one or both could be 0). 53. For the basis step 
(n = 2) the first person cuts the cake into two portions that 
she thinks are each 1/2 of the cake, and the second person 
chooses the portion he thinks is at least 1/2 of the cake (at 
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least one of the pieces must satisfy that condition). For the 
inductive step, suppose there are k + 1 people. By the induc- 
tive hypothesis, we can suppose that the first k people have 
divided the cake among themselves so that each person is sat- 
isfied that he got at least a fraction 1/k of the cake. Each of 
them now cuts his or her piece into k + 1 pieces of equal size. 
The last person gets to choose one piece from each of the first 
k people’s portions. After this is done, each of the first k peo- 
ple is satisfied that she still has (1 /k)(k/(K+ 1)) = 1/(k+ 1) of 
the cake. To see that the last person is satisfied, suppose that 
he thought that the ith person (1 < i < k) had a portion p; of 
the cake, where x, pi = 1. By choosing what he thinks is 
the largest piece from each person, he is satisfied that he has 
at least DS p;/(k + 1) = (1/(k + D) DE, p; = 1/(k + 1) of 
the cake. 55. We use the notation (i, j) to mean the square in 
row i and column j and use induction on i + j to show that ev- 
ery square can be reached by the knight. Basis step: There 
are six base cases, for the cases when i + j < 2. The 
knight is already at (0, 0) to start, so the empty sequence of 
moves reaches that square. To reach (1, 0), the knight moves 
(0,0) > (2, 1) > (0, 2) > (1, 0). Similarly, to reach (0, 1), 
the knight moves (0, 0) > (1, 2) > (2,0) > (0, 1). Note that 
the knight has reached (2, 0) and (0, 2) in the process. For the 
last basis step there is (0,0) > (1, 2) > (2,0) > (0,1) > 
(2, 2) > (0,3) > (1, 1). Inductive step: Assume the inductive 
hypothesis, that the knight can reach any square (i, j) for which 
i +j =k, where k is an integer greater than 1. We must show 
how the knight can reach each square (i, j) wheni+j=k-+1. 
Because k + 1 > 3, at least one of i and j is at least 2. If i > 2, 
then by the inductive hypothesis, there is a sequence of moves 
ending at (i—2, j+ 1), because i—2+j+1 =i+j—1=k; from 


there it is just one step to (i, j), similarly, if 7 > 2. 57. Basis 
step: The base cases n = 0 andn = 1 are true because 
the derivative of x° is 0 and the derivative of x! = x is 1. 


Inductive step: Using the product rule, the EAS hypoth- 
esis, and the basis step shows that 2 xl = La. x*) = 

xe Dyk + xh fx=x- kek] 4 xk. i eae Guin 
59. Basis step: Fork = 0,1 = 1 (mod m). Inductive step: 
Suppose that a = b (mod m) and a‘ = b* (mod m); we must 
show that att! = b+! (mod m). By Theorem 5 from Sec- 
tion 4.1, a -a¥ = b-b* (mod m), which by definition says that 
a+! = b+! (mod m). 61. Let P(n) be “[(py > pr) A (P2 > 
P3) A = A Prai > Pal > [P1 A A Pra) > Prl” 
Basis step: P(2) is true because (p) > p) > (Pi > Po) 
is a tautology. Inductive step: Assume P(k) is true. To show 
[P1 > P) A A Phi > Ph A Pk > Prowl > 
[(P1 ^ AP- ^P) > P41 lis a tautology, assume that the hy- 
pothesis of this conditional statement is true. Because both the 
hypothesis and P(k) are true, it follows that (p; A = A Pz-1) > 
P, is true. Because this is true, and because p, —> p+ is true 
(it is part of the assumption) it follows by hypothetical syllo- 
gism that (p, A---Apy_1) > P41 18 true. The weaker statement 
(pi Av APk-1 APE) > Pry follows from this. 63. We will 
first prove the result when n is a power of 2, that is, if n = Qk 
k = 1,2, .... Let P(k) be the statement A > G, where A and G 
are the arithmetic and geometric means, respectively, of a set 


S-32 Answers to Odd-Numbered Exercises 


of n = 2* positive real numbers. Basis step: k = 1 andn = 
2! = 2. Note that (4/a; — Vay > 0. Expanding this shows 
that a, — 2,/a,a, + a > 0, that is, (a, + ay)/2 > (aja). 
Inductive step: Assume that P(k) is true, with n = 2k, We 
will show that P(k + 1) is true. We have 2‘+! = 2n. Now 
(ay + ay + = + dy) /(2n) = [Cay +a +o +.4,)/ + yg + 
an2 + + a„)/n]/2 and similarly (ajaz = a,)'/0) = 
[(ay = a)!" (an1 + a3,)!/"]!/. To simplify the notation, let 
A(x, y, ... ) and G(x, y, ... ) denote the arithmetic mean and 
geometric mean of x, y, ..., respectively. Also, if x < x’, 
y < y’, and so on, then A(x, y,...) < A(x, y’, ...) and 
Gx, y...) < G, y', ... ). Hence, A(q, ...,a,) = 
A(A(Qy, ..., ap), Alan <-> An) 2 A(G(ay, ..., Ay), 
Gläpib o n) 2 G(G(Gy, ss Ay), Gär a án) = 
G(a;, ..., 4y,). This finishes the proof for powers of 2. Now 
if n is not a power of 2, let m be the next higher power of 


2, and let a,,), ..., Am all equal A(a,,...,a,) = a. Then 
we have [(a,a,-*-a,)a” "]'/" < A(ay, ...,4,,), because m is 
a power of 2. Because A(a), ...,d,,) = a, it follows that 
(a, ---a,)/mal-"/™" < a!" Raising both sides to the (m/n)th 


power gives G(d,...,4,) < A(dj,...,4,). 65. Basis step: 
For n = 1, the left-hand side is just n which is 1. For n = 2, 
there are three nonempty subsets {1}, {2}, and {1, 2}, so the 


left-hand side is : + ; + = = 2. Inductive step: Assume that 
the statement is true for k. The set of the first k + 1 positive 
integers has many nonempty subsets, but they fall into three 
categories: a nonempty subset of the first k positive integers 
together with k + 1, a nonempty subset of the first k positive 
integers, or just {k +1}. By the inductive hypothesis, the sum 
of the first category is k. For the second category, we can fac- 
tor out 1/(k+ 1) from each term of the sum and what remains 
is just k by the inductive hypothesis, so this part of the sum is 
k/(k + 1). Finally, the third category simply yields 1/(k + 1). 
Hence, the entire summation is k+k/(k+1)+1/(k+1) =k+1. 
67. Basis step: If A, © A>, then A, satisfies the condition of 
being a subset of each set in the collection; otherwise A, C A}, 
so A, satisfies the condition. Inductive step: Assume the in- 
ductive hypothesis, that the conditional statement is true for 
k sets, and suppose we are given k + 1 sets that satisfy the 
given conditions. By the inductive hypothesis, there must be 
a set A; for some i < k such that A; € A; forl < j < k. 
If A; © A;,,,, then we are done. Otherwise, we know that 
Ay, © A; and this tells us that A,,, satisfies the condition 
of being a subset of Aj for 1 <j <k+1. 69.G(1) = 0, 
G(2) = 1, G3) = 3, G44) = 4 71. To show that 2n — 4 
calls are sufficient to exchange all the gossip, select persons 1, 
2, 3, and 4 to be the central committee. Every person outside 
the central committee calls one person on the central commit- 
tee. At this point the central committee members as a group 
know all the scandals. They then exchange information among 
themselves by making the calls 1-2, 3-4, 1-3, and 2-4 in that 
order. At this point, every central committee member knows 
all the scandals. Finally, again every person outside the cen- 
tral committee calls one person on the central committee, at 
which point everyone knows all the scandals. [The total num- 


ber of calls is (n—4)+4+(n—4) = 2n—4.] That this cannot be 
done with fewer than 2n — 4 calls is much harder to prove; see 
Sandra M. Hedetniemi, Stephen T. Hedetniemi, and Arthur L. 
Liestman, “A survey of gossiping and broadcasting in com- 
munication networks,” Networks 18 (1988), no. 4, 319-349, 
for details. 73. We prove this by mathematical induction. 
The basis step (n = 2) is true tautologically. Forn = 3, 
suppose that the intervals are (a, b), (c, d), and (e, f), where 
without loss of generality we can assume thata < c < e. 
Because (a, b) N (e, f) # Ø, we must have e < b; fora 
similar reason, e < d. It follows that the number halfway 
between e and the smaller of b and d is common to all three 
intervals. Now for the inductive step, assume that whenever 
we have k intervals that have pairwise nonempty intersections 
then there is a point common to all the intervals, and suppose 


that we are given intervals J,, J, ..., 1,4, that have pairwise 
nonempty intersections. For each i from 1 to k, let J; = LOL41- 
We claim that the collection J4, Jy, ..., Ją satisfies the induc- 


tive hypothesis, that is, that Ji, N Ji, # Ø for each choice of 
subscripts i, and i,. This follows from the n = 3 case proved 
above, using the sets J; , Z; , and Z441. We can now invoke the 
inductive hypothesis to conclude that there is a number com- 
mon to all of the sets J; fori = 1, 2, ..., k, which perforce 
is in the intersection of all the sets J; fori = 1,2,...,k + 1. 
75. a) The basis step is E- j/0 + 1)! < 1, which is true 
because the left-hand side equals 1/2. It does not follow from 
Ye J/G +1)! < 1 that Yt i/G + 1)! < 1 because the 
amount by which YI + 1)! is less than 1 could be less 
than (k + 1)/(k + 2)!, the new term being added in. b) Basis 
step is the true statement that 1/2! < 1 — 1/2!. Assuming in- 
ductive hypothesis P(k) that Xi T/G+)! < 1-1/4 1}, it 
follows that i"! j/G+1)! = Le, j/G+ D!+ +) /k+2)! < 
1-1/(K+ 1 !4+ (K+ 1)/(K +2)! = 1-(K4+2)/(K +2)! 4+ (K+ 
1)/(k +2)! = 1-1/(K+2)!, whichis P(k+ 1). 77. Pair up 
the people. Have the people stand at mutually distinct small 
distances from their partners but far away from everyone else. 
Then each person throws a pie at his or her partner, so every- 
one gets hit. 


79. 


81. Let P(n) be the statement that every 2” x 2” x 2” checker- 
board with a 1 x 1 x 1 cube removed can be covered by tiles 
that are 2 x 2 x 2 cubes each with a 1 x 1 x 1 cube removed. 
The basis step, P(1), holds because one tile coincides with the 
solid to be tiled. Now assume that P(k) holds. Now consider 
a 21 x 21 x 21 cube with a 1 x 1 x 1 cube removed. Split 
this object into eight pieces using planes parallel to its faces 
and running through its center. The missing 1 x 1 x 1 piece 
occurs in one of these eight pieces. Now position one tile with 


its center at the center of the large object so that the missing 
1 x 1 X 1 cube lies in the octant in which the large object is 
missing a 1 x 1 X 1 cube. This creates eight 2‘ x 2% x 2* cubes, 
each missing a 1 X 1 x 1 cube. By the inductive hypothesis 
we can fill each of these eight objects with tiles. Putting these 
tilings together produces the desired tiling. 


“i 
Pip 
fo 


85. Let Q(n) be P(n + b — 1). The statement that P(n) is true 
forn = b,b + 1,b +2, ... is the same as the statement that 
Q(m) is true for all positive integers m. We are given that P(b) 
is true [1.e., that Q(1) is true], and that P(k) > P(k + 1) for all 
k > b [i.e., that Q(m) > Q(m + 1) for all positive integers m]. 
Therefore, by the principle of mathematical induction, Q(m) 
is true for all positive integers m. 


Be 


Section 5.2 


1. Basis step: We are told we can run one mile, so P(1) is 
true. Inductive step: Assume the inductive hypothesis, that we 
can run any number of miles from 1 to k. We must show that 
we can run k + | miles. If k = 1, then we are already told 
that we can run two miles. If k > 1, then the inductive hy- 
pothesis tells us that we can run k — 1 miles, so we can run 
(kK-1)+2=k+I1 miles. 3.a) P(8) is true, because we can 
form 8 cents of postage with one 3-cent stamp and one 5-cent 
stamp. P(9) is true, because we can form 9 cents of postage 
with three 3-cent stamps. P(10) is true, because we can form 
10 cents of postage with two 5-cent stamps. b) The state- 
ment that using just 3-cent and 5-cent stamps we can form j 
cents postage for all j with 8 < j < k, where we assume 
that k > 10 c) Assuming the inductive hypothesis, we can 
form k + 1 cents postage using just 3-cent and 5-cent stamps 
d) Because k > 10, we know that P(k — 2) is true, that is, that 
we can form k—2 cents of postage. Put one more 3-cent stamp 
on the envelope, and we have formed k + 1 cents of postage. 
e) We have completed both the basis step and the inductive 
step, so by the principle of strong induction, the statement is 
true for every integer n greater than or equal to 8. 5. a) 4, 
8, 11, 12, 15, 16, 19, 20, 22, 23, 24, 26, 27, 28, and all values 
greater than or equal to 30 b) Let P(n) be the statement that 
we can form n cents of postage using just 4-cent and 1 1-cent 
stamps. We want to prove that P(n) is true for all n > 30. For 
the basis step, 30 = 11 +11 +4+ 4. Assume that we can form 
k cents of postage (the inductive hypothesis); we will show 
how to form k + 1 cents of postage. If the k cents included 
an 1 1-cent stamp, then replace it by three 4-cent stamps. Oth- 
erwise, k cents was formed from just 4-cent stamps. Because 
k > 30, there must be at least eight 4-cent stamps involved. 
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Replace eight 4-cent stamps by three | 1-cent stamps, and we 
have formed k + 1 cents in postage. c) P(n) is the same as in 
part (b). To prove that P(n) is true for all n > 30, we check for 
the basis step that 30 = 114+11+4+4, 31 = 11+4+4+4+4+4, 
32 = 4+4+4+4+4+4+4+4, and 33 = 11+11+11. For the in- 
ductive step, assume the inductive hypothesis, that P(/) is true 
for all j with 30 < j < k, where k is an arbitrary integer greater 
than or equal to 33. We want to show that P(k + 1) is true. Be- 
cause k—3 > 30, we know that P(k—3) is true, that is, that we 
can form k—3 cents of postage. Put one more 4-cent stamp on 
the envelope, and we have formed k + 1 cents of postage. In 
this proof, our inductive hypothesis was that P(/) was true for 
all values of j between 30 and k inclusive, rather than just that 
P(30) was true. 7. We can form all amounts except $1 and 
$3. Let P(n) be the statement that we can form n dollars using 
just 2-dollar and 5-dollar bills. We want to prove that P(n) is 
true for all n > 5. (It is clear that $1 and $3 cannot be formed 
and that $2 and $4 can be formed.) For the basis step, note that 
5 = 5 and 6 = 2+2+2. Assume the inductive hypothesis, that 
P(j) is true for all j with 5 <j < k, where k is an arbitrary in- 
teger greater than or equal to 6. We want to show that P(k + 1) 
is true. Because k — 1 > 5, we know that P(k — 1) is true, that 
is, that we can form k — 1 dollars. Add another 2-dollar bill, 
and we have formed k + 1 dollars. 9. Let P(n) be the state- 
ment that there is no positive integer b such that V2 = n/b. 
Basis step: P(1) is true because V2 > 1 > 1/b for all posi- 
tive integers b. Inductive step: Assume that P(j) is true for all 
j < k, where k is an arbitrary positive integer; we prove that 
P(k+1) is true by contradiction. Assume that V2 =(k+1)/b 
for some positive integer b. Then 2b? = (k + 1}, so (k + 1)? 
is even, and hence, k + 1 is even. So write k + 1 = 2t for 
some positive integer t, whence 2b? = 4/° and b? = 217. By 
the same reasoning as before, b is even, so b = 2s for some 
positive integer s. Then V2 = (k + 1)/b = (21)/(2s) = t/s. 
But ż < k, so this contradicts the inductive hypothesis, and 
our proof of the inductive step is complete. 11. Basis step: 
There are four base cases. If n = 1 = 4-0 + 1, then clearly 
the second player wins. If there are two, three, or four matches 
(n=4-04+2,n =4-0+4+3, orn = 4-1), then the first player can 
win by removing all but one match. Inductive step: Assume 
the strong inductive hypothesis, that in games with k or fewer 
matches, the first player can win if k = 0,2, or 3 (mod 4) and 
the second player can win if k = 1 (mod 4). Suppose we have 
a game with k + 1 matches, with k > 4. Ifk+1=0 (mod 4), 
then the first player can remove three matches, leaving k — 2 
matches for the other player. Because k — 2 = 1 (mod 4), by 
the inductive hypothesis, this is a game that the second player 
at that point (who is the first player in our game) can win. Sim- 
ilarly, if k + 1 = 2 (mod 4), then the first player can remove 
one match; and if k + 1 = 3 (mod 4), then the first player 
can remove two matches. Finally, if k + 1 = 1 (mod 4), then 
the first player must leave k, k — 1, or k — 2 matches for the 
other player. Because k = 0 (mod 4), k—1 =3 (mod 4), and 
k—2 =2 (mod 4), by the inductive hypothesis, this is a game 
that the first player at that point (who is the second player 
in our game) can win. 13. Let P(n) be the statement that 
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exactly n — 1 moves are required to assemble a puzzle with n 
pieces. Now P(1) is trivially true. Assume that P(j) is true for 
allj < k, and consider a puzzle with k + 1 pieces. The final 
move must be the joining of two blocks, of size j and k+ 1 — j 
for some integer j with 1 < j < k. By the inductive hypoth- 
esis, it required j — 1 moves to construct the one block, and 
k+1-—j-—1=k-—j moves to construct the other. Therefore, 
1+(G-—1)+(k—-/j) = k moves are required in all, so P(k + 1) is 
true. 15. Let the Chomp board have n rows and n columns. 
We claim that the first player can win the game by making the 
first move to leave just the top row and leftmost column. Let 
P(n) be the statement that if a player has presented his oppo- 
nent with a Chomp configuration consisting of just n cookies 
in the top row and n cookies in the leftmost column, then he 
can win the game. We will prove VnP(n) by strong induction. 
We know that P(1) is true, because the opponent is forced to 
take the poisoned cookie at his first turn. Fix k > 1 and assume 
that P(j) is true for all j < k. We claim that P(k+ 1) is true. It is 
the opponent’s turn to move. If she picks the poisoned cookie, 
then the game is over and she loses. Otherwise, assume she 
picks the cookie in the top row in column j, or the cookie in 
the left column in row j, for some j with 2 < j < k + 1. The 
first player now picks the cookie in the left column in row j, 
or the cookie in the top row in column j, respectively. This 
leaves the position covered by P(j — 1) for his opponent, so 
by the inductive hypothesis, he can win. 17. Let P(n) be the 
statement that if a simple polygon with n sides is triangulated, 
then at least two of the triangles in the triangulation have two 
sides that border the exterior of the polygon. We will prove 
Vn > 4P(n). The statement is true for n = 4, because there 
is only one diagonal, leaving two triangles with the desired 
property. Fix k > 4 and assume that P(j) is true for all j with 
4 <j < k. Consider a polygon with k+1 sides, and some trian- 
gulation of it. Pick one of the diagonals in this triangulation. 
First suppose that this diagonal divides the polygon into one 
triangle and one polygon with k sides. Then the triangle has 
two sides that border the exterior. Furthermore, the k-gon has, 
by the inductive hypothesis, two triangles that have two sides 
that border the exterior of that k-gon, and only one of these 
triangles can fail to be a triangle that has two sides that bor- 
der the exterior of the original polygon. The only other case is 
that this diagonal divides the polygon into two polygons with 
j sides and k + 3 — j sides for some j with 4 < j < k- 1. 
By the inductive hypothesis, each of these two polygons has 
two triangles that have two sides that border their exterior, 
and in each case only one of these triangles can fail to be a 
triangle that has two sides that border the exterior of the orig- 
inal polygon. 19. Let P(n) be the statement that the area of 
a simple polygon with n sides and vertices all at lattice points 
is given by /(P) + B(P)/2 — 1. We will prove P(n) for all 
n > 3. We begin with an additivity lemma: If P is a simple 
polygon with all vertices at lattice points, divided into poly- 
gons P, and P, by a diagonal, then /(P) + B(P)/2 —- 1 = 
U(Pi) + B(P,)/2 — 1] + U(P}) + B(P})/2 — 1]. To prove 
this, suppose there are k lattice points on the diagonal, not 
counting its endpoints. Then /(P) = I(P,) + I(P,) + k and 


B(P) = B(P,) + B(P,) — 2k — 2; and the result follows by 
simple algebra. What this says in particular is that if Pick’s 
formula gives the correct area for P, and P}, then it must give 
the correct formula for P, whose area is the sum of the areas 
for P and P,; and similarly if Pick’s formula gives the cor- 
rect area for P and one of the P,’s, then it must give the correct 
formula for the other P;. Next we prove the theorem for rect- 
angles whose sides are parallel to the coordinate axes. Such 
a rectangle necessarily has vertices at (a, b), (a, c), (d, b), and 
(d, c), where a, b, c, and d are integers with b < c anda < d. 
Its area is (c — b)(d — a). Also, B = 2(c — b + d — a) and 
I=(c-b-1)\(d—-a-1)= (c-b)Xd-a)-(c-b)-(d-a)+1. 
Therefore, /+B/2—1 = (c—b)(d—a)—-(c—b)—-(d—-a)+1+(c— 
b+d-a)-— 1 = (c —b)(d — a), which is the desired area. Next 
consider a right triangle whose legs are parallel to the coordi- 
nate axes. This triangle is half a rectangle of the type just con- 
sidered, for which Pick’s formula holds, so by the additivity 
lemma, it holds for the triangle as well. (The values of B and 7 
are the same for each of the two triangles, so if Picks’s formula 
gave an answer that was either too small or too large, then it 
would give a correspondingly wrong answer for the rectan- 
gle.) For the next step, consider an arbitrary triangle with ver- 
tices at lattice points that is not of the type already considered. 
Embed it in as small a rectangle as possible. There are several 
possible ways this can happen, but in any case (and adding 
one more edge in one case), the rectangle will have been par- 
titioned into the given triangle and two or three right triangles 
with sides parallel to the coordinate axes. Again by the addi- 
tivity lemma, we are guaranteed that Pick’s formula gives the 
correct area for the given triangle. This completes the proof of 
P(3), the basis step in our strong induction proof. For the in- 
ductive step, given an arbitrary polygon, use Lemma 1 in the 
text to split it into two polygons. Then by the additivity lemma 
above and the inductive hypothesis, we know that Pick’s for- 
mula gives the correct area for this polygon. 21. a) In the 
left figure Zabp is smallest, but bp is not an interior diago- 
nal. b) In the right figure bd is not an interior diagonal. c) In 
the right figure bd is not an interior diagonal. 23. a) When 
we try to prove the inductive step and find a triangle in each 
subpolygon with at least two sides bordering the exterior, it 
may happen in each case that the triangle we are guaranteed 
in fact borders the diagonal (which is part of the boundary of 
that polygon). This leaves us with no triangles guaranteed to 
touch the boundary of the original polygon. b) We proved the 
stronger statement Vn > 47(n) in Exercise 17. 25. a) The 
inductive step here allows us to conclude that P(3), P(5), ... 

are all true, but we can conclude nothing about P(2), P(4), .... 
b) P(n) is true for all positive integers n, using strong induc- 
tion. c) The inductive step here enables us to conclude that 
P(2), P(4), P(8), P(16), ...are all true, but we can conclude 
nothing about P(n) when n is not a power of 2. d) This is 
mathematical induction; we can conclude that P(n) is true for 
all positive integers n. 27. Suppose, for a proof by contra- 
diction, that there is some positive integer n such that P(n) is 
not true. Let m be the smallest positive integer greater than 
n for which P(m) is true; we know that such an m exists 


because P(m) is true for infinitely many values of m. But we 
know that P(m) > P(m — 1), so P(m — 1) is also true. Thus, 
m — 1 cannot be greater than n, so m — 1 = n and P(n) is in 
fact true. This contradiction shows that P(n) is true for all n. 
29. The error is in going from the base case n = 0 to the next 
case, n = 1; we cannot write 1 as the sum of two smaller 
natural numbers. 31. Assume that the well-ordering prop- 
erty holds. Suppose that P(1) is true and that the conditional 
statement [P(1) A P(2) A + A P(n)] > P(n + 1) is true for 
every positive integer n. Let S be the set of positive integers 
n for which P(n) is false. We will show S = Ø. Assume that 
S + Ø. Then by the well-ordering property there is a least 
integer m in S. We know that m cannot be 1 because P(1) is 
true. Because n = mis the least integer such that P(n) is false, 
P(1), P(2), ..., P(m — 1) are true, and m — 1 > 1. Because 
[PC) A P(2) A + A P(m — 1)] > Pm) is true, it follows 
that P(m) must also be true, which is a contradiction. Hence, 
S=§. 33. In each case, give a proof by contradiction based 
on a “smallest counterexample,” that is, values of n and k such 
that P(n, k) is not true and n and k are smallest in some sense. 
a) Choose a counterexample with n + k as small as possible. 
We cannot haven = l andk = 1, because we are given 
that P(1, 1) is true. Therefore, either n > 1 ork > 1. In the 
former case, by our choice of counterexample, we know that 
P(n—1, k) is true. But the inductive step then forces P(n, k) to 
be true, a contradiction. The latter case is similar. So our sup- 
position that there is a counterexample must be wrong, and 
P(n, k) is true in all cases. b) Choose a counterexample with 
n as small as possible. We cannot have n = 1, because we 
are given that P(1, k) is true for all k. Therefore, n > 1. By 
our choice of counterexample, we know that P(n — 1, k) is 
true. But the inductive step then forces P(n, k) to be true, a 
contradiction. c) Choose a counterexample with k as small 
as possible. We cannot have k = 1, because we are given 
that P(n, 1) is true for all n. Therefore, k > 1. By our choice 
of counterexample, we know that P(n, k — 1) is true. But the 
inductive step then forces P(n, k) to be true, a contradiction. 
35. Let P(n) be the statement that if x,, x5, ..., x, are n dis- 
tinct real numbers, then n — 1 multiplications are used to find 
the product of these numbers no matter how parentheses are 
inserted in the product. We will prove that P(n) is true using 
strong induction. The basis case P(1) is true because 1—1 = 0 
multiplications are required to find the product of x}, a product 
with only one factor. Suppose that P(x) is true for 1 < k < n. 
The last multiplication used to find the product of the n + 1 
distinct real numbers x), Xz, ..., X,,X,4, 18 a multiplication of 
the product of the first k of these numbers for some k and the 
product of the last n + 1 — k of them. By the inductive hy- 
pothesis, k — | multiplications are used to find the product of 
k of the numbers, no matter how parentheses were inserted in 
the product of these numbers, and n — k multiplications are 
used to find the product of the other n + 1 — k of them, no 
matter how parentheses were inserted in the product of these 
numbers. Because one more multiplication is required to find 
the product of all n + 1 numbers, the total number of multi- 
plications used equals (k — 1) + (n — k) + 1 = n. Hence, 
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P(n + 1) is true. 37. Assume that a = dq +r = dq' + r' 
withO < r < dandO < r’ < d. Thend(q-q’) = r' =r. 
It follows that d divides r’ — r. Because -d < r’ —r < d, 
we have r’ — r = 0. Hence, 7’ = r. It follows that q = g'. 
39. This is a paradox caused by self-reference. The answer 
is “no.” There are a finite number of English words, so only 
a finite number of strings of 15 words or fewer; therefore, 
only a finite number of positive integers can be so described, 
not all of them. 41. Suppose that the well-ordering prop- 
erty were false. Let S be a nonempty set of nonnegative in- 
tegers that has no least element. Let P(n) be the statement 
“i é Sfor i= 0, 1, ..., n.” P(O) is true because if 0 € S then 
S has a least element, namely, 0. Now suppose that P(n) is 
true. Thus, 0 ¢ S, 1 € S, ..., n € S. Now, n + 1 cannot be in 
S, for if it were, it would be its least element. Thus, P(n + 1) 
is true. So by the principle of mathematical induction, n ¢ S 
for all nonnegative integers n. Thus, S = Ø, a contradiction. 
43. Strong induction implies the principle of mathematical in- 
duction, for if one has shown that P(k) —> P(k + 1) is true, 
then one has also shown that [P(1) A -+ A P(k)] > P(k + 1) is 
true. By Exercise 41, the principle of mathematical induction 
implies the well-ordering property. Therefore, by assuming 
strong induction as an axiom, we can prove the well-ordering 


property. 
Section 5.3 


1.a) f(1) = 3,f2) =5,f3) = 7,74 =9 b) fd) = 3, 
f = 9,fG) = 27,f4) = 81 9f) = 2, fQ) = 4, 
f3) = 16, f(4) = 65,536 d) f(1) = 3, f(2) = 13, f(3) = 183, 
S(4 = 33,673 3. a) f(2) = -—1,f(3) = 5,f(4) = 2,f(5) = 17 
b) f(2) = —4, f(3) = 32, f(4) = —4096, f(5) = 536,870,912 
c) f(2) = 8, f(3) = 176, f(4) = 92,672, f(5) = 25,764, 174,848 
DF) = -—5,f@) =—4,f4) = z f5) = -32 5. a) Not 
valid b) f(n) = 1 —n. Basis step: f(0) = 1 = 1 — 0. Inductive 
step: if f(k) = 1—k, thenf(k+1)=f(K)-1=1-k-1=1 

(kK+1). eo) f(n) = 4-nifn > 0, and f(0) = 2. Basis step: 
f0) = 2 and f(1) = 3 = 4 — 1. Inductive step (with k > 1): 
fk+1) =f®-1 = 4-H-1 = 4-41). 
d)f(n) = 2'+D/1, Basis step: f(0) = 1 = 210+D/I 
and f(1) = 2 = 2!0+D/2], Inductive step (with k > 1): 
f(k +1)= 2f(k -1)= Di 21k/2] = 2lk/2]+1 = DLCk+1)+1)/2] 
e)f(n) = 3”. Basis step: Trivial. Inductive step: For odd n, 
fn) = 3f(n — 1) = 3-3"! = 3"; and forevenn > 1, 
f(n) = 9f(n — 2) = 9 - 3"? = 3”, 7. There are many pos- 
sible correct answers. We will supply relatively simple ones. 
a) d,,, =4,+6forn > landa, =6b)a,,,; =a,+2forn > 1 
anda, =3 c)a,,, = 10a, forn > 1 anda, = 10d)a,,,; =a, 
forn > landa; =5 9. F(0) =0, F(x) = F(n — 1) + n for 
n> 1 11. P„(0) = 0,P,(n +1) = Pan) +m 13.Let 
P(n) be “fi +f + + fon- = fon” Basis step: P(1) is true 
because f, = 1 = fọ. Inductive step: Assume that P(k) is true. 
Then fi +f + 0 + Sora thongs = Sar tSas = Sorta thoes: 
15. Basis step: fof, +f =0-1+1-1=1° = f?. I- 
ductive step: Assume that fof, + fifa + = + faih = A 
Then fof, + ff +0 + Sardak + Padari + for = 
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in + fahri + frervforso = Sor Gre + fori) + Sarria = 
Sahara + Porh = Ce + fordra = a 17. The 
number of divisions used by the Euclidean algorithm to find 
gcd fa) is O forn = 0,1 forn = 1, andn — 1 
for n > 2. To prove this result for n > 2 we use math- 
ematical induction. For n = 2, one division shows that 
gcd(fz, fa) = ged(2, 1) = ged(1, 0) = 1. Now assume that k— 1 
divisions are used to find gcd(f}41, fp). To find gcd(fi4., fk+1) 
first divide f,,» by f,,; to obtain fiyo = 1- fi, +f. After 
one division we have gcd (fp42 fz41) = gcdi.41,f,). By the in- 
ductive hypothesis it follows that exactly k — 1 more divisions 
are required. This shows that k divisions are required to find 
gcd(fi2,f;41), finishing the inductive proof. 19. |A] = —1. 


Hence, |A”| = (—1)”. It follows that f,.f,_1 -f = (-1)". 
21. a) Proof by induction. Basis step: For n = 1, max(—a,) = 
—a; = —min(a,). For n = 2, there are two cases. If a, > a}, 
then —a, > —a,, so max(—a,, —a,) = —a; = — min(a}, a). 
Ifa, < a, then -a; < —a,, so max(—a,, —d,) = 
—a, = -— min(a,, a). Inductive step: Assume true for k 
with k > 2. Then max(—a,, —dy, ..., -ap ap) = 
max(max(—d,, ..., —d,), —@,4,) = max(— min(q,, ..., ap) 
—4y4,) = —min(min(ay, ..., a), ag) = — min(a;,..., 


441). b) Proof by mathematical induction. Basis step: For 
n = 1, the result is the identity a, +b, = a,+b,. For n = 2, first 
consider the case in which a, + b; > a + by. Then max(a, + 
by, dy+by) = a, +b,. Also note that a; < max(a}, ay) and b; < 
max(b,, b2), soa; +b; < max(a,, a,)+max(b,, b,). Therefore, 
max(a; +1, dy +b) =a, +b, < max(aj, ay) + max(h,, b3). 
The case with a, + b} < a, + by is similar. In- 
ductive step: Assume that the result is true for k. Then 
max(a, + Dy, dy + b, ..., ap + bp Aggy + bi) = 
max(max(a, + bj, dy + by, ..., ap + dD), ay + 
bya) < max(max(a;, dy, ..., ay) + max(b,, bz, ..., dy), 
ak1 + bu) <  max(max(a;, dy, ... , ap) Ap) + 
max(max(h,, by, ..., by), b41) = max(a}, dy, ..., Ap Aggy) + 
max(h,, by, ... , by, b41). ©) Same as part (b), but replace 
every occurrence of “max” by “min” and invert each inequal- 
ity. 23.5€S,andx+yeSifxyeS. 25.a)0€S, and 
ifx E€ S, then x+2 €Sandx—2€S. b)2 €S,andifx € S, 
thenx+3€S.c) 1ES,2€5,3 €S,4€S,andifx E S, then 
x+5 ES. 27. a) Basis step: 5 = 5 (mod 10). Inductive 
step: If n = 5 (mod 10), then 3n = 3-5 = 15 = 5 (mod 10) 
and n? = 5* = 25 = 5 (mod 10). b) 35 ¢ S because 35 is 
not a multiple of 3 nora perfect square. 29. a) (0, 1), (1, 1), 
(2, 1); (0, 2), C1, 2), (2, 2), (3, 2), (4 2); (0, 3), (1, 3), (2, 3), 
(3, 3), (4, 3), (5, 3), (6, 3); (0, 4), CL 4), (2, 4), G, 4), (4 4), 
(5, 4), (6, 4), (7, 4), (8, 4) b) Let P(n) be the statement that 
a < 2b whenever (a, b) € S is obtained by n applications of 
the recursive step. Basis step: P(0) is true, because the only 
element of S obtained with no applications of the recursive 
step is (0, 0), and indeed 0 < 2 - 0. Inductive step: Assume 
that a < 2b whenever (a, b) € S is obtained by k or fewer 
applications of the recursive step, and consider an element ob- 
tained with k + 1 applications of the recursive step. Because 
the final application of the recursive step to an element (a, b) 
must be applied to an element obtained with fewer applica- 


tions of the recursive step, we know that a < 2b. Add 0 < 2, 
1 < 2,and2 < 2, respectively, to obtaina < 2(b + 1), 
a+1<2(b+1),anda+2 < 2(b+ 1), as desired. c) This 
holds for the basis step, because 0 < 0. If this holds for (a, b), 
then it also holds for the elements obtained from (a, b) in the 
recursive step, because adding 0 < 2,1 < 2,and2 < 2, 
respectively, to a < 2b yields a < 2(b + 1),a+1<2(b+1), 
anda +2 < 2(b + 1). 31. a) Define S by (1, 1) € S, and 
if (a, b) E S, then (a + 2, b) € S, (a b +2) E€ S, and 
(a+ 1, b+ 1) € S. All elements put in S satisfy the condition, 
because (1, 1) has an even sum of coordinates, and if (a, b) has 
an even sum of coordinates, then so do (a + 2, b), (a, b + 2), 
and (a + 1, b + 1). Conversely, we show by induction on the 
sum of the coordinates that if a + b is even, then (a, b) € S. 
If the sum is 2, then (a, b) = (1, 1), and the basis step put 
(a, b) into S. Otherwise the sum is at least 4, and at least one 
of (a—2, b), (a, b—2), and (a— 1, b — 1) must have positive in- 
teger coordinates whose sum is an even number smaller than 
a + b, and therefore must be in S. Then one application of the 
recursive step shows that (a, b) € S. b) Define S by (1, 1), 
(1, 2), and (2, 1) are in S, and if (a, b) E S, then (a + 2, b) and 
(a, b +2) are in S. To prove that our definition works, we note 
first that (1, 1), (1, 2), and (2, 1) all have an odd coordinate, 
and if (a, b) has an odd coordinate, then so do (a + 2, b) and 
(a, b+2). Conversely, we show by induction on the sum of the 
coordinates that if (a, b) has at least one odd coordinate, then 
(a, b) € S. If (a, b) = (1, 1) or (a, b) = (1, 2) or (a, b) = (2, 1), 
then the basis step put (a, b) into S. Otherwise either a or b is 
at least 3, so at least one of (a — 2, b) and (a, b — 2) must have 
positive integer coordinates whose sum is smaller than a + b, 
and therefore must be in S. Then one application of the recur- 
sive step shows that (a, b) € S. c) (1, 6) € S and (2, 3) E S, 
and if (a, b) € S, then (a + 2, b) € S and (a,b + 6) E S. 
To prove that our definition works, we note first that (1, 6) and 
(2, 3) satisfy the condition, and if (a, b) satisfies the condition, 
then so do (a + 2, b) and (a, b + 6). Conversely we show by 
induction on the sum of the coordinates that if (a, b) satisfies 
the condition, then (a, b) € S. For sums 5 and 7, the only 
points are (1, 6), which the basis step put into S, (2, 3), which 
the basis step put into S, and (4,3) = (2 + 2, 3), which is 
in S by one application of the recursive definition. For a sum 
greater than 7, eithera > 3, ora < 2 andb > 9, in which 
case either (a — 2, b) or (a, b — 6) must have positive integer 
coordinates whose sum is smaller than a + b and satisfy the 
condition for being in S. Then one application of the recursive 
step shows that (a,b) E S. 33. If x is a set or a variable rep- 
resenting a set, then x is a well-formed formula. If x and y are 
well-formed formulae, then so are x, (xUy), (xNy), and (x—y). 
35.a)Ifx ED = {0, 1, 2,3,4,5, 6, 7, 8, 9}, then m(x) = x; 
if s = tx, where t E D* and x E D, then m(s) = min(m(s), x). 
b) Lett = wx, wherew E€ D* andx € D.Ifw = A, 
then m(st) = m(sx) = min(m(s), x) = min(m(s), m(x)) by 
the recursive step and the basis step of the definition of m. 
Otherwise, m(st) = m((sw)x) = min(m(sw), x) by the 
definition of m. Now m(sw) = min(m(s), m(w)) by the in- 
ductive hypothesis of the structural induction, so m(st) = 


min(min(m(s), m(w)), x) =  min(m(s), min(m(w), x)) by 
the meaning of min. But min(@m(w), x) = m(wx) = m(t) 
by the recursive step of the definition of m. Thus, m(st) = 
min(m(s), m(t)). 37. AÈ = A and (ux)® = xu? for x € È, 
u € X*, 39.w? = Aand w"! = ww”. 41. When the 
string consists of n Os followed by n 1s for some nonnegative 
integer n 43. Let P(i) be “l~w') = i- (w).” P(O) is true 
because (w?) = 0 = 0 - (w). Assume P(i) is true. Then 
lw!) = ww!) = lw) + Ww’) = Ww) +:i- Kw) = (i +1) - Kw). 
45. Basis step: For the full binary tree consisting of just a root 
the result is true because n(T) = 1 and A(T) = 0, and 
1 > 2-0+ 1. Inductive step: Assume that n(T,) > 2h(T,) + 1 
and n(T,) > 2h(T,) + 1. By the recursive definitions of n(T) 
and A(T), we have n(T) = 1 + n(T,) + n(T,) and h(T) = 
1+ max(h(T,), h(T,)). Therefore, n(T) = 1+n(T,) +n(Ty) = 
1+2h(T,)+14+2A(T)+1> 14+2-max(A(T,), A(T>)) +2 = 
1+2(max(A(T,), (T>) +1) = 1+24(T). 47. Basis step: 
ago = 0 = 0 +0. Inductive step: Assume that ay „ = m' +n! 
whenever (m’, n’) is less than (m, n) in the lexicographic or- 
dering of NXN. Ifn = 0 then an = am-1n t1 = m-l+n+1 = 
m+n. Ifn > 0, then apn = 4nn-1 +1 =m+n-l+1=m+n. 
49. a) Pm = P, because a number exceeding m cannot be 
used in a partition of m. b) Because there is only one way to 
partition 1, namely, 1 = 1, it follows that P} „ = 1. Because 
there is only one way to partition m into Is, P,,, = 1. When 
n > mit follows that P,,,,, = Pm m because a number exceed- 
ing m cannot be used. P,,,,, = 1 + Pm because one extra 
partition, namely, m = m, arises when m is allowed in the par- 
tition. Pin = Pmn-1 +P if m > n because a partition of 
m into integers not exceeding n either does not use any ns and 
hence, is counted in P,,,,_; or else uses an n and a partition of 
m — n, and hence, is counted in P,,,_,,. €) Ps = 7, P = 11 
51. Let P(n) be “A(n, 2) = 4.” Basis step: P(1) is true be- 
cause A(1, 2) = A(O, AC, 1)) = AQO,2) = 2-2 = 4. 
Inductive step: Assume that P(n) is true, that is, A(n, 2) = 4. 
Then A(n + 1,2) = A(n, A(n + 1,1) = Al, 2) = 4. 
53. a) 16 b) 65,536 55. Use a double induction argument 
to prove the stronger statement: A(m, k) > A(m, I) when 
k > I. Basis step: When m = O the statement is true be- 
cause k > l implies that A(0, k) = 2k > 2l = AÇ, D. 
Inductive step: Assume that A(m, x) > A(m, y) for all non- 
negative integers x and y with x > y. We will show that this 
implies that A(m + 1, k) > A(m + 1, Difk > L. Basis 
steps: When! = O and k > 0, A(m + 1,2) = 0 and either 
A(m + 1, k) = 2 or A(m + 1, k) = A(m, A(m + 1, k — 1)). If 
m = 0, this is 2A(1, k — 1) = 2*. If m > O, this is greater than 
0 by the inductive hypothesis. In all cases, A(m + 1, k) > 0, 
and in fact, A(m + 1, k) > 2.Ifl = l andk > 1, then 
A(m + 1,1) = 2 and A(m + 1, k) = A(m, A(m + 1, k — 1)), 
with A(m + 1, k — 1) > 2. Hence, by the inductive hypoth- 
esis, A(m, A(m + 1, k — 1)) > A(m,2) > A(m, 1) = 2. 
Inductive step: Assume that A(m + 1, r) > A(m + 1, s) for all 
r> s,s =0,1,...,1 Thenifk+ 1 > 1+ 1 it follows that 
A(m+1,k +1) = A(m, A(m + 1, k)) > A(m, A(m + 1, k)) = 
A(m+1,/+1). 57. From Exercise 56 it follows that A (i, j) > 
A(i—1,j) > = > A0, =2j >j. 59. Let P(n) be “F(n) is 
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well defined.” Then P(0) is true because F(0) is specified. As- 
sume that P(A) is true for all k < n. Then F(n) is well defined 
at n because F(n) is given in terms of F(0), F(1), ..., F(n— 1). 
So P(n) is true for all integers n. 61. a) The value of F(1) 
is ambiguous. b) F(2) is not defined because F(0) is not de- 
fined. c) F(3) is ambiguous and F(4) is not defined because 
FÉ) makes no sense. d) The definition of F(1) is ambigu- 
ous because both the second and third clause seem to apply. 
e) F(2) cannot be computed because trying to compute F(2) 
gives F(2)=1+F(F(1))=1+F(2). 63.a)1 b)2 œ)3 

d)3 e)4 4 g)5 65. f(n) = [n/a] 67. f3 (n) = 
[loglogn] for n > 2, ff) = 0 


Section 5.4 
1. First, we use the recursive step to write 5! = 5 - 4!. We 
then use the recursive step repeatedly to write 4! = 4 - 3!, 


3! = 3-2!,2! = 2. 1!, and 1! = 1 - O!. Inserting the value 
of 0! = 1, and working back through the steps, we see that 
I!=1-1=1,2! =2-1!=2-1=2,3!=3-2!=3-2=6, 
4! = 4-3! = 4-6 = 24, and 5! = 5-4! = 5-24 = 120. 3. With 
this input, the algorithm uses the else clause to find that 
gcd(8, 13) = gcd(13 mod 8, 8) = gcd(5, 8). It uses this clause 
again to find that gcd(5, 8) = gcd(8 mod 5,5) = gcd(3, 5), 
then to get gcd(3, 5) = gcd(5 mod 3, 3) = gcd(2, 3), then 
gcd(2, 3) = gcd(3 mod 2, 2) = gcd(1, 2), and once more to 
get gcd(1, 2) = gcd(2 mod 1, 1) = gcd(0, 1). Finally, to find 
gcd(0, 1) it uses the first step with a = 0 to find that gcd(0, 1) = 
1. Consequently, the algorithm finds that gcd(8, 13) = 1. 
5. First, because n = 11 is odd, we use the else clause to 
see that mpower(3, 11,5) = (mpower (3, 5, 5) mod 5 - 
3 mod 5) mod 5. We next use the else clause again to see 
that mpower (3, 5,5) =  (mpower(3, 2, 5)? mod 5 - 3 
mod 5) mod 5. Then we use the else if clause to see 
that mpower (3, 2,5) =  mpower(3, 1, 5)? mod 5. Us- 
ing the else clause again, we have mpower(3, 1,5) = 
(mpower (3, 0, 5)? mod 5-3 mod 5) mod 5. Finally, using 
the if clause, we see that mpower (3, 0,5) = 1. Working 
backward it follows that mpower (3, 1,5) = (1? mod 5 - 
3 mod 5) mod 5 = 3, mpower(3, 2,5) = 3? mod5 = 
4, mpower (3, 5,5) = (42 mod 5-3 mod 5) mod 5 = 3, and 
finally mpower (3, 11,5) = (3? mod 5-3 mod 5) mod 5 = 
2. We conclude that 3!! mod 5 = 2. 
7. procedure mult(n: positive integer, x: integer) 
if n = 1 then return x 
else return x + mult (n — 1, x) 
9. procedure sum of odds(n: positive integer) 
ifn = 1 then return 1 
else return sum of odds (n—1)+2n-1 
11. procedure smallest(a,, ..., a„: integers) 
ifn = 1 then return a, 
else return 
min(smallest (ay, ..., A,_1)s A) 
13. procedure modfactorial(n, m: positive integers) 
if n = 1 then return 1 
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else return 
(n - modfactorial(n — 1, m)) mod m 

15. procedure gcd(a, b: nonnegative integers) 

{a < b assumed to hold} 

if a = 0 then return b 

else if a = b — a then return a 

else if a < b — a then return gcd(a, b — a) 

else return gcd(b — a, a) 
17. procedure multiply(x, y: nonnegative integers) 

if y = 0 then return 0 

else if y is even then 

return 2 - multiply (x, y/2) 

else return 2 - multiply (x, (y—1)/2) + x 
19. We use strong induction on a. Basis step: If a = 0, we 
know that gcd(0, b) = b for all b > 0, and that is precisely 
what the if clause does. Inductive step: Fix k > 0, assume 
the inductive hypothesis—that the algorithm works correctly 
for all values of its first argument less than k—and consider 
what happens with input (k, b), where k < b. Because k > 0, 
the else clause is executed, and the answer is whatever the 
algorithm gives as output for inputs (b mod k, k). Because 
b mod k < k, the input pair is valid. By our inductive hy- 
pothesis, this output is in fact gcd(b mod k, k), which equals 
gcd(k, b) by Lemma 1 in Section 4.3. 21.Ifn = 1, then 
nx = x, and the algorithm correctly returns x. Assume that 
the algorithm correctly computes kx. To compute (k + 1)x it 
recursively computes the product of k + 1 — 1 = k and x, 
and then adds x. By the inductive hypothesis, it computes that 
product correctly, so the answer returned is kx + x = (k + 1)x, 
which is correct. 
23. procedure square(n: nonnegative integer) 

if n = 0 then return 0 

else return square (n— 1)+2(n—1)+ 1 
Let P(n) be the statement that this algorithm correctly com- 
putes n?. Because 0? = 0, the algorithm works correctly 
(using the if clause) if the input is 0. Assume that the algo- 
rithm works correctly for input k. Then for input k + 1, it 
gives as output (because of the else clause) its output when 
the input is k, plus 2(k + 1 — 1) + 1. By the inductive 
hypothesis, its output at k is k*, so its output at k + 1 is 
KR +2(k+1-1)+1 =K +2k4+1 = (k+ 1)’, as desired. 
25. n multiplications versus 2” 27. O(log n) versus n 
29. procedure a(n: nonnegative integer) 

if n = 0 then return 1 

else if n = 1 then return 2 

else return a(n — 1) - a(n — 2) 
31. Iterative 


33. procedure iferative(n: nonnegative integer) 
ifn = 0 then z := 1 
else if n = 1 then z := 2 


else 
X= 
y= 
zi=3 


fori :=1ton—2 
w:=x+y+z 


x:=y 
y:=z 
z:=w 


return z {z is the nth term of the sequence} 


35. We first give a recursive procedure and then an iterative 
procedure. 

procedure r(n: nonnegative integer) 

if n < 3 then return 2n + 1 

else return r(n — 1) - (r(n — 2))? - (r(n — 3)? 


procedure ¿(n: nonnegative integer) 
ifn = 0 then z := 1 
else if n = 1 then z := 3 


else 
Pl 
y:=3 
zi=5 


for i := 1ton—2 
wisz-yex 


x:=y 
y:=z 
z= w 


return z {z is the nth term of the sequence} 
The iterative version is more efficient. 


37. procedure reverse(w: bit string) 
n := length(w) 
if n < 1 then return w 
else return 
substr(w, n, n)reverse (substr (w, 1, n — 1)) 
{substr(w, a, b) is the substring of w consisting of 
the symbols in the ath through bth positions} 


39. The procedure correctly gives the reversal of A as A (ba- 
sis step), and because the reversal of a string consists of its 
last character followed by the reversal of its first n — 1 char- 
acters (see Exercise 37 in Section 5.3), the algorithm behaves 
correctly when n > 0 by the inductive hypothesis. 41. The 
algorithm implements the idea of Example 14 in Section 5.1. 
If n = 1 (basis step), place the one right triomino so that its 
armpit corresponds to the hole in the 2 x 2 board. If n > 1, 
then divide the board into four boards, each of size 2”! x2"7!, 
notice which quarter the hole occurs in, position one right tri- 
omino at the center of the board with its armpit in the quarter 
where the missing square is (see Figure 7 in Section 5.1), and 
invoke the algorithm recursively four times—once on each of 
the 2”-! x 2”-1 boards, each of which has one square missing 
(either because it was missing to begin with, or because it is 
covered by the central triomino). 


43. procedure A(m, n: nonnegative integers) 
if m = 0 then return 2n 
else if n = 0 then return 0 
else if n = 1 then return 2 
else return A(m — 1, A(m, n — 1)) 


45. bdafghzpok 
bdafg a Fee 
oe b fg -i 5 ok 
I, JN. JN LN 


\/ 


a x we 
N 


EN 


ew 


47. Let the two lists be 1, 2, ..., m — 1,m + n — 1 and 
m,m + l, ...,m +n -— 2,m +n, respectively. 49. Ifn = 1, 
then the algorithm does nothing, which is correct because a 
list with one element is already sorted. Assume that the algo- 
rithm works correctly for n = 1 through n = k. Ifn =k +1, 
then the list is split into two lists, L} and L,. By the inductive 
hypothesis, mergesort correctly sorts each of these sublists; 
furthermore, merge correctly merges two sorted lists into one 
because with each comparison the smallest element in L; UL, 
not yet put into Lis put there. 51. O(n) 53.6 55. O(n’) 


Section 5.5 


1. Suppose that x = 0. The program segment first assigns the 
value | to y and then assigns the value x + y = 0 + 1 = 1 to 
z. 3. Suppose that y = 3. The program segment assigns the 
value 2 to x and then assigns the value x + y = 2 +3 = 5 to z. 
Because y = 3 > 0 it then assigns the value z+ 1 = 5+ 1 = 6 
to z. 

5. (p^ condition1){S,}q 

(pA acondition| A condition2){S,}q 


(pA acondition| A acondition2 
+ A acondition(n — 1)){S,,}q 
p{if condition\ then S$}; 
else if condition2 then S,;...; else S„}q 

7. We will show that p = “power = ae Tandi<n+ 1” isa 
loop invariant. Note that p is true initially, because before the 
loop starts, i = 1 and power = 1 = x? = x'-!. Next, we must 
show that if p is true and i < n after an execution of the loop, 
then p remains true after one more execution. The loop incre- 
ments i by 1. Hence, because 7 < n before this pass, i < n + 1 
after this pass. Also the loop assigns power -x to power. By the 
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inductive hypothesis we see that power is assigned the value 
xil -x = x, Hence, p remains true. Furthermore, the loop 
terminates after n traversals of the loop with i = n + 1 be- 
cause i is assigned the value 1 prior to entering the loop, is 
incremented by | on each pass, and the loop terminates when 
i > n. Consequently, at termination power = x", as desired. 
9. Suppose that p is “m and n are integers.” Then if the con- 
dition n < 0 is true, a = —n = |n| after S, is executed. If the 
condition n < 0 is false, then a = n = |n] after S, is executed. 
Hence, p{S,}q is true where q is p A (a = |n|). Because S, 
assigns the value 0 to both k and x, it is clear that q{ S, }r is 
true where r is q A (k = 0) A (x = 0). Suppose that r is true. 
Let P(k) be “x = mk and k < a.” We can show that P(k) is a 
loop invariant for the loop in S,. P(Q) is true because before 
the loop is entered x = 0 = m-0 and 0 < a. Now assume P(k) 
is true and k < a. Then P(k + 1) is true because x is assigned 
the value x +m = mk +m = m(k + 1). The loop terminates 


when k = a, and at that point x = ma. Hence, r{S3}s is 
true where s is “a = |n| and x = ma.” Now assume that s 
is true. Then if n < 0 it follows that a = —n, so x = —mn. 


In this case S, assigns —x = mn to product. If n > 0 then 
x = ma = mn, so S, assigns mn to product. Hence, s{S4}t 
is true. 11. Suppose that the initial assertion p is true. Then 
because p{S}qo is true, gp is true after the segment S is exe- 
cuted. Because gy — q; is true, it also follows that q; is true 
after S is executed. Hence, p{S}q, is true. 13. We will use 
the proposition p, “gcd(a, b) = gcd(x, y) and y > 0,” as the 
loop invariant. Note that p is true before the loop is entered, 
because at that point x = a, y = b, and y is a positive in- 
teger, using the initial assertion. Now assume that p is true 
and y > 0; then the loop will be executed again. Inside the 
loop, x and y are replaced by y and x mod y, respectively. By 
Lemma 1 of Section 4.3, gcd(x, y) = gcd(y, x mod y). There- 
fore, after execution of the loop, the value of gcd(x, y) is the 
same as it was before. Moreover, because y is the remainder, 
it is at least 0. Hence, p remains true, so it is a loop invari- 
ant. Furthermore, if the loop terminates, then y = 0. In this 
case, we have gcd(x, y) = x, the final assertion. Therefore, the 
program, which gives x as its output, has correctly computed 
gcd(a, b). Finally, we can prove the loop must terminate, be- 
cause each iteration causes the value of y to decrease by at 
least 1. Therefore, the loop can be iterated at most b times. 


Supplementary Exercises 


1. Let P(n) be the statement that this equation holds. Basis 
step: P(1) says 2/3 = 1 — (1/31), which is true. Inductive 
step: Assume that P(k) is true. Then 2/3 +2/9+2/27 +--+ 
2/3" +2/3"+! = 1-1/3" +2/3"*! (by the inductive hypoth- 
esis), and this equals 1 — 1/3”+!, as desired. 3. Let P(n) be 
“1-142-24--4n-2""! = (n—1)2"+1.” Basis step: P(A) is 
true because 1-1 = 1 = (1—1)2! +1. Inductive step: Assume 
that P(k) is true. Then 1-14+2-2+---+k-2!!4(k+1)-2* = 
(kK—1)2k 414 (K+ 1)2* = 2k-2* 4:1 =[(kK +1) —1]24! +1. 
5. Let P(n) be “1/(1-4)+---+1/[Gn—2)3n+1)] = n/(3n+1).” 
Basis step: P(1) is true because 1/(1 - 4) = 1/4. Inductive 
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step: Assume P(k) is true. Then 1/(1 - 4) + ++ + 1/[(3k - 
2)(3k + 1)] + 1/[GK + GK + 4)] = k/(3k + 1) +1/[Bk + 
NGbk + A] = [kBk + 4) + 1/[Bk + DBA + 4] = 
[3k + Ik + DI/IBA + DBA + 4] = (k + 1D/BK + 4). 
7. Let P(n) be “2” > n>.” Basis step: P(10) is true because 
1024 > 1000. Inductive step: Assume P(k) is true. Then 
(K+ 1% = 43k? 43k41<562 49h <P 4+R =2Kh < 
2.2k = 2! 9. Let P(n) be “a — b is a factor of a” — b"? 
Basis step: P(1) is trivially true. Assume P(k) is true. Then 
kt! — pkt! = qkt! —abk +ab* — pk+! = alat =b") +bk(a —b). 
Then because a — b is a factor of a‘ — bk and a — b is a fac- 
tor of a — b, it follows that a — b is a factor of a*t! — btt! 
11. Basis step: When n = 1, 6"*! + 7"-! = 3647 = 43. 
Inductive step: Assume the inductive hypothesis, that 43 di- 
vides 6”+! +77"! ; we must show that 43 divides 6"? + 72"t!, 
We have 6"*7 + 77"+! = 6-6"! +49 . 771 = 6.6"! 4+6. 
Prt4 43.7! = 6(6"+1 +777!) +43 - 77-1, By the induc- 
tive hypothesis the first term is divisible by 43, and the second 
term is divisible by 43; therefore, the sum is divisible by 43. 
13. Let P(n) be “a+(a+d)+---+(a+nd) = (n+ 1)\(2a+nd)/2.” 
Basis step: P(1) is true because a + (a + d) = 2a + d = 
2(2a + d)/2. Inductive step: Assume that P(k) is true. Then 
a + (a + d)+ = + (a + kd) + [a + (k + Dd] = 
(k + 1)(2a + kd)/2 +a + (k + 1)d = +(2ak + 2a + kd + 
kd+2a+2kd+2d) = +(2ak + 4a+ kd + 3kd + 2d) = s(k+ 
2)[2a + (k+ 1)d]. 15. Basis step: This is true for n = 1 be- 
cause 5/6 = 10/12. Inductive step: Assume that the equation 
holds for n = k, and consider n = k+1. Then g = an = 
l i(i+1)(i+2) 
5t i+4 k+5 _ k(3k+7) k+5 


i=1 i(i+1)(i+2) (KAI (K+2)k+3) — 2(k+1)(k+2) ; ee ea) 
i i 1 — š + +. = 
(by the inductive hypothesis) = ENEA C oe 


1 = td 
2(k+1)(K+2)(k+3) i [k(3k t 7) eae 2(k + 5)] ~ 2(k+1)(K+2)(k+3) 
3 2 A Z . Ds 


1 . - - — (KDG (k+1)+7) : 
2(k+2)(k+3) Gk + 10)k + 1) = 2(k+1)+1)((k+1) +2)” as desired. 


17. Basis step: The statement is true for n = 1 because the 
derivative of g(x) = xe” is x - e” + e* = (x + 1)e* by the prod- 
uct rule. Inductive step: Assume that the statement is true for 
n = k, i.e., the kth derivative is given by g = (x + kbe". 
Differentiating by the product rule gives the (k + 1)st deriva- 
tive: g) = (x + het +e = [x + (k + De’, as de- 
sired. 19. We will use strong induction to show that f, is 
even if n = 0 (mod 3) and is odd otherwise. Basis step: This 
follows because fọ = 0 is even and f} = 1 is odd. Inductive 
step: Assume that if j < k, then f; is even if j = 0 (mod 3) 
and is odd otherwise. Now suppose k + 1 = 0 (mod 3). Then 
Seat = fe + fe- is even because f, and f,_; are both odd. If 
k+1 = 1 (mod 3), then fz41 = fg +f; is odd because f, 
is even and f,_, is odd. Finally, if k + 1 = 2 (mod 3), then 
Seat = Se + fp is odd because f, is odd and f,_; is even. 
21. Let P(n) be the statement that ff, + frathar = Snake 
for every nonnegative integer k. Basis step: This consists of 
showing that P(O) and P(1) both hold. P(O) is true because 
Sifo + Serif = feri 9 +S 1 =f. Because fifi +f = 
Sie + fear = fean it follows that P(1) is true. Inductive step: 
Now assume that P(j) holds. Then, by the inductive hypoth- 


esis and the recursive definition of the Fibonacci numbers, it 
follows that fis ifs theahjve = th) thaiG +h = 
Ffi- + firth) + id tferhir) = Silk tive = Fisk: 
This shows that PG + 1) is true. 23. Let P(n) be the state- 
ment is + E + +Ê = 11,4; + 2. Basis step: P(0) and 
P(1) both hold because ° = 2?= 2-142 = hl, +2 
and Ê + É = 2+1? = 1-342 = hh +2 h- 
ductive step: Assume that P(k) holds. Then by the inductive 
hypothesis È + É + = +2404, = hk t2+ = 
Dra Alig +2 = liyi lg42 +2. This shows that P(k+ 1) holds. 
25. Let P(n) be the statement that the identity holds for the in- 
teger n. Basis step: P(1) is obviously true. Inductive step: As- 
sume that P(k) is true. Then cos((k + 1)x) + isin((k + 1)x) = 
cos(kx + x) + isin(kx + x) = cos kx cos x — sin kx sin x + 
i(sin kx cos x + cos kx sin x) = cos x(cos kx + isin kx)(cos x + 
isin x) = (cosx+isinx)*(cos x+i sin x) = (cosx+isinx)*!. It 
follows that P(k+1)istrue. 27. Rewrite the right-hand side 
as 2+! (n? -2n+3)—6. Forn = 1 we have 2 = 4-2—6. Assume 
that the equation holds for n = k, and consider n = k + 1. 
Then Da Pa = pa Po + (k + 122k = 2+ Ge 
2k + 3) — 6 + (k? + 2k + 1)2'+! (by the inductive hypo- 
thesis) = HKR + 4) -6 = XR +2) -6 = 
220k + 1} — 2k + 1) + 3] — 6. 29. Let P(n) be the 
statement that this equation holds. Basis step: In P(2) both 
sides reduce to 1/3. Inductive step: Assume that P(k) is true. 
Then 2i 1/ -1) = (i lg = D) + 1/[(k + 
1)? — 1] = (k — 1)Gk + 2)/[4k(K + 1)] + 1/[(k + 1} - 1] 
by the inductive hypothesis. This simplifies to (k — 1)(3k + 
2)/[4k(k+1)]+1/(K?+2k) = (3K +5k?)/[4k(k+ 1)(k+2)] = 
{I(k+1)— 1][3(k+1)+2]}/[4(k+ 1)(k+2)], which is exactly 
what P(k + 1) asserts. 31. Let P(n) be the assertion that at 
least n + 1 lines are needed to cover the lattice points in the 
given triangular region. Basis step: P(0) is true, because we 
need at least one line to cover the one point at (0, 0). Inductive 
step: Assume the inductive hypothesis, that at least k+ 1 lines 
are needed to cover the lattice points with x > 0, y > 0, and 
x+ y < k. Consider the triangle of lattice points defined by 
x > 0,y > 0O,andx+y < k+ 1. By way of contradic- 
tion, assume that k + 1 lines could cover this set. Then these 
lines must cover the k + 2 points on the line x +y = k+ 1. 
But only the line x + y = k + 1 itself can cover more than 
one of these points, because two distinct lines intersect in at 
most one point. Therefore, none of the k + 1 lines that are 
needed (by the inductive hypothesis) to cover the set of lattice 
points within the triangle but not on this line can cover more 
than one of the points on this line, and this leaves at least one 
point uncovered. Therefore, our assumption that k + 1 lines 
could cover the larger set is wrong, and our proof is complete. 
33. Let P(n) be Bt = MA*M™!. Basis step: Part of the given 
conditions. Inductive step: Assume the inductive hypothesis. 
Then B+! = BB‘ = MAM'Bk = MAMMAM! (by 
the inductive hypothesis) = MAIA*M-! = MAAM”! = 
MA“'!M-!. 35. We prove by mathematical induction the 
following stronger statement: For every n > 3, we can write 
n! as the sum of n of its distinct positive divisors, one of which 


is 1. That is, we can write n! = a, +a)+-+-+a,,, where each a; 
is a divisor of n!, the divisors are listed in strictly decreasing 
order, and a, = 1. Basis step: 3! = 3 +2 + 1. Inductive step: 
Assume that we can write k! as a sum of the desired form, say 
k! = a, + a) + +++ + a, where each a; is a divisor of n!, the 
divisors are listed in strictly decreasing order, anda, = 1. 
Consider (k + 1)!. Then we have (k + 1)! = (k + Dk! = 
(k + 1a +a, ++ +a) = (K+ Ia, + (K+ Way + + 
(k+ Da, = (k+ Da, + (k+ Da, + +k- ata. 
Because each a; was a divisor of k!, each (k + 1)a; is a divisor 
of (k + 1)!. Furthermore, k - a, = k, which is a divisor of 
(k + 1)!, and a, = 1, so the new last summand is again 1. 
(Notice also that our list of summands is still in strictly de- 
creasing order.) Thus, we have written (k + 1)! in the desired 
form. 37. Whenn = | the statement is vacuously true. As- 
sume that the statement is true for n = k, and consider k + 1 
people standing in a line, with a woman first and a man last. 
If the kth person is a woman, then we have that woman stand- 
ing in front of the man at the end. If the kth person is a man, 
then the first k people in line satisfy the conditions of the in- 
ductive hypothesis for the first k people in line, so again we 
can conclude that there is a woman directly in front of a man 
somewhere in the line. 39. Basis step: When n = 1 there 
is one circle, and we can color the inside blue and the outside 
red to satisfy the conditions. Inductive step: Assume the in- 
ductive hypothesis that if there are k circles, then the regions 
can be 2-colored such that no regions with a common bound- 
ary have the same color, and consider a situation with k + 1 
circles. Remove one of the circles, producing a picture with 
k circles, and invoke the inductive hypothesis to color it in 
the prescribed manner. Then replace the removed circle and 
change the color of every region inside this circle. The result- 
ing figure satisfies the condition, because if two regions have 
a common boundary, then either that boundary involved the 
new circle, in which case the regions on either side used to be 
the same region and now the inside portion is different from 
the outside, or else the boundary did not involve the new cir- 
cle, in which case the regions are colored differently because 
they were colored differently before the new circle was re- 
stored. 41. If n = 1 then the equation reads 1-1 = 1-2/2, 
which is true. Assume that the equation is true for n and con- 
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43. Let T(n) be the statement that the sequence of towers of 2 
is eventually constant modulo n. We use strong induction to 
prove that T(n) is true for all positive integers n. Basis step: 
When n = 1 (and n = 2), the sequence of towers of 2 modulo 
nis the sequence of all Os. Inductive step: Suppose that k is an 
integer with k > 2. Suppose that T(j) is true for 1 <j <k-1. 
In the proof of the inductive step we denote the rth term of the 
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sequence modulo n by a,. First suppose k is even. Let k = 2°q 
where s > 1 andq < kis odd. When j is large enough, dj. 2S, 
and for such j, a; = 2" isa multiple of 2°. It follows that for 
sufficiently large j, a; = 0 (mod 2°). Hence, for large enough i, 
25 divides a;,; — a;. By the inductive hypothesis 7(q) is true, 
so the sequence a), a>, a}, ... is eventually constant modulo q. 
This implies that for large enough i, q divides a,;,, — a;. Be- 
cause gcd(q, 2°) = 1 and for sufficiently large i both q and 
25 divide a; — a;,k = 2°q divides a,,, — a; for sufficiently 
large i. Hence, for sufficiently large i, a;,, — a; = 0 (mod k). 
This means that the sequence is eventually constant modulo k. 
Finally, suppose k is odd. Then gcd(2, k) = 1, so by Euler’s 
theorem (found in elementary number theory books, such as 
[Ro10]), we know that 2@ = 1 (mod k). Let r = (k). Be- 
cause r < k, by the inductive hypothesis T(r), the sequence 
dy, A, az, ... is eventually constant modulo r, say equal to c. 
Hence, for large enough i, for some integer t;, a; = tr + c. 
Hence, a,,, = 2% = 2'/*¢ = (2")'2° = 2° (mod k). This shows 
that a), a), ... is eventually constant modulo k. 45. a) 92 
b)91 c)91 d)91 e)91 f)91 47. The basis step is 
incorrect because n # 1 forthe sum shown. 49. Let P(n) be 
“the plane is divided into n?—n+2 regions by n circles if every 
two of these circles have two common points but no three have 
a common point.” Basis step: P(1) is true because a circle di- 
vides the plane into 2 = 17—1+2 regions. Inductive step: As- 
sume that P(K) is true, that is, k circles with the specified prop- 
erties divide the plane into k? — k + 2 regions. Suppose that a 
(k+1)st circle is added. This circle intersects each of the other 
k circles in two points, so these points of intersection form 2k 
new arcs, each of which splits an old region. Hence, there are 
2k regions split, which shows that there are 2k more regions 
than there were previously. Hence, k + 1 circles satisfying the 
specified properties divide the plane into k?—k+2+2k = (k?+ 
2k+1)—(k+1)42 = (k+1)°-(k+1)+2 regions. 51. Suppose 
V2 were rational. Then V2 = a/b, where a and b are positive 
integers. It follows that the set S = {ny2 |ne N}ANisa 
nonempty set of positive integers, because by2 = a belongs 
to S. Let t be the least element of S, which exists by the well- 
ordering property. Then t = sy2 for some integer s. We have 
t-s= sVy2 -s= s( v2 — 1), so t— s is a positive integer 
because y2 > 1. Hence, t — s belongs to S. This is a contra- 
diction because t — $ = sv2 —s < s. Hence, y2 is irrational. 
53.a)Letd = gcd(a}, dy, ..., a,). Then d is a divisor of 
each a; and so must be a divisor of gcd(a,,_ |, 4„). Hence, d is 
a common divisor of a}, az, ..., @,_», and gcd(a,- a,). To 
show that it is the greatest common divisor of these numbers, 
suppose that c is acommon divisor of them. Then c is a divi- 
sor of a; fori = 1,2, ...,n — 2 anda divisor of gcd(a,_j, d,,), 
so it is a divisor of a,_, and a,,. Hence, c is a common divi- 
sor Of a}, a, ...,4,_,, and a,,. Hence, it is a divisor of d, the 
greatest common divisor of a4, az, ..., d,,. It follows that d is 
the greatest common divisor, as claimed. b)Ifn = 2, ap- 
ply the Euclidean algorithm. Otherwise, apply the Euclidean 
algorithm to a,_, and a,, obtaining d = gcd(a,_,, a,,), and 
then apply the algorithm recursively to a), do, ... , A,_2, d. 
55.f(n) = n°. Let P(n) be “f(n) = n??? Basis step: P1) 
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is true because f(1) = 1 = 12, which follows from the 
definition of f. Inductive step: Assume f(n) = n°. Then 
fat) = f(mt+1) - 1)4+2M4+1)-1 = f+ 
2n +1 =n” +2n+1 = (n+ 1%. 57.a) 4,0, 1,00, 01, 
11, 000, 001, 011, 111, 0000, 0001, 0011, 0111, 1111, 00000, 
00001, 00011, 00111, 01111, 11111 b) S = {af | æis a string 
of m Os and £ is a string of n ls, m >0,n > 0} 59. Apply 
the first recursive step to A to get () € B. Apply the second 
recursive step to this string to get 00 € B. Apply the first 
recursive step to this string to get (OQ) € B. By Exercise 62, 
(())) is not in B because the number of left parentheses does 
not equal the number of right parentheses. 61. A, O, (0), OO 
63.a)0 b)-2 ¢)2 d)O 


65. 
procedure generate(n: nonnegative integer) 
if n is odd then 
S := S(n — 1) {the S constructed by generate(n — 1)} 
T := T(n — 1) {the T constructed by generate(n — 1)} 
else if n = 0 then 
S:=@ 
T := {A} 
else 
S’ := S(n — 2) {the S constructed by generate(n — 2)} 
T’ := T(n — 2) {the T constructed by generate(n — 2)} 
T:=T'U {|x €T' US’ A length(x) =n —2} 
S:=S'U {xy|x €T’ Ay E€T'US’ A length(xy) = n} 
{T US is the set of balanced strings of length at most n} 
67. If x < y initially, then x := y is not executed, sox < y is a 
true final assertion. If x > y initially, then x := y is executed, 
so x < y is again a true final assertion. 
69. procedure zerocount(d,, dy, ..., a„: list of integers) 
if n = 1 then 
if a, = 0 then return | 
else return 0 


else 
if a, = 0 then return zerocount (dy, do, ..., ap) + 1 
else return zerocount (a4, dy, ..., Gy_1) 


71. We will prove that a(n) is a natural number and a(n) < n. 
This is true for the base case n = 0 because a(0) = 0. Now 
assume that a(n — 1) is a natural number and a(n — 1) <n-1. 
Then a(a(n — 1)) is a applied to a natural number less than 
or equal to n — 1. Hence, a(a(n — 1)) is also a natural num- 
ber less than or equal to n — 1. Therefore, n — a(a(n — 1)) 
is n minus some natural number less than or equal to n — 1, 
which is a natural number less than or equal ton. 73. From 
Exercise 72, a(n) = |(n + l)u] and a(n — 1) = [np]. Be- 
cause y < 1, these two values are equal or they differ by 1. 
First suppose that un — |an] < 1 — n. This is equivalent to 
u(n+1)< 1+ |an]. If this is true, then [w(n+1)| = |an]. On 
the other hand, if wn—|yn| > 1—y, then w(n+1) > 14+ | pn], 
so |w(n+1)| = |wn|+1,asdesired. 75. f(0) = 1, m(0) = 0; 
fŒ) = 1, m1) = 0; f(2) = 2, m(2) = 1; f3) = 2, m8) = 2; 
fA) = 3,m4) = 2; f5) = 3, m5) = 3; f(6) = 4, 
m(6) = 4; f(T) = 5, m(7) = 4; f(8) = 5, m(8) = 5; f(9) = 6, 
m9) = 6 77. The last occurrence of n is in the position 
for which the total number of Is, 2s, ... , ns all together is that 


position number. But because a, is the number of occurrences 
of k, this is just pe d,, as desired. Because f(n) is the sum 
of the first n terms of the sequence, f(f(m)) is the sum of the 
first f(n) terms of the sequence. But because f(n) is the last 
term whose value is n, this means that the sum is the sum of 
all terms of the sequence whose value is at most n. Because 
there are a, terms of the sequence whose value is k, this sum 
is Dio K'ap as desired. 


CHAPTER 6 


Section 6.1 


1. a)5850 b)343 3.a)4!? b) 51? 5.42 7.26? 
9.676 11.28 13.n + 1 (counting the empty string) 
15. 475,255 (counting the empty string) 17. 1,321,368,961 
19.a) 729 b)256 c) 1024 d)64 21.a) Seven: 56, 63, 
70, 77, 84, 91, 98 b) Five: 55, 66, 77, 88, 99 c) One: 
77 23.a)128 b)450 c)9 d)675_~ e) 450 f) 450 
g)225 h)75 25.a)990  b) 500 027 27. 3°° 
29. 52,457,600 31. 20,077,200 33. a) 37,822,859,361 
b) 8,204,716,800 c) 40,159,050,880 d) 12,113,640,000 
e) 171,004,205,215 f) 72,043,541,640 g) 6,230,721,635 
h) 223,149,655 35. a) 0 b) 120 c) 720 d) 2520 937. a)2 
ifn = 1,2ifn = 2,0ifn > 3 b)2”? forn > l; 
lifn = 1 c)2(n-— 1) 39.(n + 1)" 41. If 7 is even, 
2/2: if nis odd, 2”+D/2 43.a) 175 b)248 c) 232 d) 84 
45.40 47.60 49.a)240 b)480  c)360 51.352 
53. 147 55.33 57. a) 9,920,671,339,261,325,541,376 = 
9.9 x 10°! b) 6,641,514,961,387,068,437,760 ~ 6.6 x 
10?! c) About 314,000 years 59. 54(646556 — 1)/63 
61. 7,104,000,000,000 63. 161°+167°+16°8 65. 666,667 
67.18 69.17 71.22 73.2n—2,2n—1 75. Let P(m) 
be the sum rule for m tasks. For the basis case take m = 2. 
This is just the sum rule for two tasks. Now assume that P(m) 
is true. Consider m + 1 tasks, T}, Ty, -Tm Ting, Which can 
be done in 1, m, ..., Nm Am1 Ways, respectively, such that 
no two of these tasks can be done at the same time. To do one 
of these tasks, we can either do one of the first m of these or 
do task T,,,,,;. By the sum rule for two tasks, the number of 
ways to do this is the sum of the number of ways to do one 
of the first m tasks, plus n,,,,;. By the inductive hypothesis, 
this isn, +n, + + +n + p41 as desired. 77. n(n — 3)/2 


m m+1? 


Section 6.2 


1. Because there are six classes, but only five weekdays, the 
pigeonhole principle shows that at least two classes must be 
held on the same day. 3.a)3 b)14 5.85 7. Because 
there are four possible remainders when an integer is divided 
by 4, the pigeonhole principle implies that given five integers, 
at least two have the same remainder. 9%. Leta, a + 1, ..., 


a + n — 1 be the integers in the sequence. The integers 
(a +i) mod n, i = 0,1, 2,..., n — 1, are distinct, because 
0 < (a+j) —-(a+k) < nwheneverO <k <j <n-l. 
Because there are n possible values for (a + i) mod n and 
there are n different integers in the set, each of these val- 
ues is taken on exactly once. It follows that there is exactly 
one integer in the sequence that is divisible by n. 11. 4951 
13. The midpoint of the segment joining the points (a, b, c) 
and (d, e, f) is (a + d)/2, (b + e)/2, (c + f)/2). It has 
integer coefficients if and only if a and d have the same 
parity, b and e have the same parity, and c and f have the 
same parity. Because there are eight possible triples of par- 
ity [such as (even, odd, even)], by the pigeonhole principle 
at least two of the nine points have the same triple of par- 
ities. The midpoint of the segment joining two such points 
has integer coefficients. 15. a) Group the first eight posi- 
tive integers into four subsets of two integers each so that the 
integers of each subset add up to 9: {1, 8}, {2, 7}, {3, 6}, 
and {4, 5}. If five integers are selected from the first eight 
positive integers, by the pigeonhole principle at least two of 
them come from the same subset. Two such integers have a 
sum of 9, as desired. b) No. Take {1, 2, 3, 4}, for example. 
17.4 19.21,251 21. a) If there were fewer than 9 fresh- 
men, fewer than 9 sophomores, and fewer than 9 juniors in the 
class, there would be no more than 8 with each of these three 
class standings, for a total of at most 24 students, contradict- 
ing the fact that there are 25 students in the class. b) If there 
were fewer than 3 freshmen, fewer than 19 sophomores, and 
fewer than 5 juniors, then there would be at most 2 freshmen, 
at most 18 sophomores, and at most 4 juniors, for a total of 
at most 24 students. This contradicts the fact that there are 25 
students in the class. 23.4, 3, 2, 1, 8,7, 6,5, 12, 11, 10, 9, 
16, 15, 14,13 25. Number the seats around the table from 1 
to 50, and think of seat 50 as being adjacent to seat 1. There are 
25 seats with odd numbers and 25 seats with even numbers. 
If no more than 12 boys occupied the odd-numbered seats, 
then at least 13 boys would occupy the even-numbered seats, 
and vice versa. Without loss of generality, assume that at least 
13 boys occupy the 25 odd-numbered seats. Then at least two 
of those boys must be in consecutive odd-numbered seats, and 
the person sitting between them will have boys as both of his 
or her neighbors. 

27. procedure long(ay, ..., a„: positive integers) 

{first find longest increasing subsequence } 
max := 0; set := 00... 00 {n bits} 


for i := 1 to 2” 
last := 0; count := 0, OK := true 
for j:= 1 ton 


if ser) = 1 then 
if a; > last then last := aj 
count := count + 1 
else OK := false 
if count > max then 
max := count 
best := set 
set := set + 1 (binary addition) 
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{max is length and best indicates the sequence } 
{repeat for decreasing subsequence with only 
changes being a; < last instead of a; > last 
and last := œ instead of last := 0} 
29. By symmetry we need prove only the first statement. Let 
A be one of the people. Either A has at least four friends, or 
A has at least six enemies among the other nine people (be- 
cause 3 +5 < 9). Suppose, in the first case, that B, C, D, 
and E are all A’s friends. If any two of these are friends with 
each other, then we have found three mutual friends. Oth- 
erwise {B, C, D, E} is a set of four mutual enemies. In the 
second case, let {B, C, D, E, F, G} be a set of enemies of A. 
By Example 13, among B, C, D, E, F, and G there are either 
three mutual friends or three mutual enemies, who form, with 
A, a set of four mutual enemies. 31. We need to show two 
things: that if we have a group of n people, then among them 
we must find either a pair of friends or a subset of n of them 
all of whom are mutual enemies; and that there exists a group 
of n — 1 people for which this is not possible. For the first 
statement, if there is any pair of friends, then the condition is 
satisfied, and if not, then every pair of people are enemies, so 
the second condition is satisfied. For the second statement, if 
we have a group of n — 1 people all of whom are enemies of 
each other, then there is neither a pair of friends nor a sub- 
set of n of them all of whom are mutual enemies. 33. There 
are 6,432,816 possibilities for the three initials and a birthday. 
So, by the generalized pigeonhole principle, there are at least 
[39,000,000/6,432,816] = 7 people who share the same ini- 
tials and birthday. 35. Because 800,001 > 200,000, the 
pigeonhole principle guarantees that there are at least two 
Parisians with the same number of hairs on their heads. The 
generalized pigeonhole principle guarantees that there are at 
least [800,001/200,000] = 5 Parisians with the same number 
of hairs on their heads. 37.18 39. Because there are six 
computers, the number of other computers a computer is con- 
nected to is an integer between 0 and 5, inclusive. However, 
0 and 5 cannot both occur. To see this, note that if some com- 
puter is connected to no others, then no computer is connected 
to all five others, and if some computer is connected to all five 
others, then no computer is connected to no others. Hence, by 
the pigeonhole principle, because there are at most five possi- 
bilities for the number of computers a computer is connected 
to, there are at least two computers in the set of six connected 
to the same number of others. 41. Label the computers C, 
through Cj 9, and label the printers P, through P,g. If we con- 
nect C, to P, fork = 1, 2,..., 20 and connect each of the 
computers C,, through Coo to all the printers, then we have 
used a total of 20 + 80 - 20 = 1620 cables. This is sufficient, 
because if computers C, through Cy, need printers, then they 
can use the printers with the same subscripts, and if any com- 
puters with higher subscripts need a printer instead of one or 
more of these, then they can use the printers that are not being 
used, because they are connected to all the printers. Now we 
must show that 1619 cables is not enough. Because there are 
1619 cables and 20 printers, the average number of comput- 
ers per printer is 1619/20, which is less than 81. Therefore, 
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some printer must be connected to fewer than 81 computers. 
That means it is connected to 80 or fewer computers, so there 
are 20 computers that are not connected to it. If those 20 com- 
puters all needed a printer simultaneously, then they would be 
out of luck, because they are connected to at most the 19 other 
printers. 43. Let a; be the number of matches completed by 
hour i. Then 1 < a, < a < ++ < aş < 125. Also 
25 < a, +24 < a, +24 < ++ < a +24 < 149. 
There are 150 numbers a}, ..., a75, a; + 24, ..., a75 +24. By 
the pigeonhole principle, at least two are equal. Because all 
the a;s are distinct and all the (a; + 24)s are distinct, it fol- 
lows that a; = a; + 24 for some i > j. Thus, in the period 
from the (j+ 1)st to the ith hour, there are exactly 24 matches. 
45. Use the generalized pigeonhole principle, placing the |S] 
objects f(s) for s € S in |T| boxes, one for each element of T. 
47. Let d; be jx— N (jx), where N( jx) is the integer closest to jx 
for 1 < j < n. Each d; is an irrational number between —1/2 
and 1/2. We will assume that n is even; the case where n is odd 
is messier. Consider the n intervals {x | j/n <x < (j+ 1)/n}, 
{x | -0 + 1)/n < x < -j/n} for j = 0, 1, ..., (n/2) — 1. If 
d; belongs to the interval {x | 0 < x < 1/ ie or to the interval 
i. | —1/n < x < 0} for some j, we are done. If not, because 
there are n—2 intervals and n numbers d,, the pigeonhole ao 
ciple tells us that there is an interval {x | (k-1)/n<x<k/n} 
containing d, and d, with r < s. The proof can be finished 
by showing that (s — r)x is within 1/n of its nearest integer. 
49. a) Assume that i, < n for all k. Then by the generalized 
pigeonhole principle, at least [(n? + 1)/n] = n + 1 of the 
numbers i), io, ... , i24] are equal. b) If a, < Ak then 
the subsequence consisting of A, followed by the i increasing 
subsequence of length i ty starting at ag contradicts the fact 
that i i = ik . Hence, ay > A+ c) If there is no increasing 
subsequence of length greater ‘than n, then parts (a) and (b) 
apply. Therefore, we have a; > ap > + > ap > apa 
decreasing sequence of length. n+l. 


Section 6.3 


1. abc, acb, bac, bca, cab, cba 3.720 5.a)120 b)720 
c)8 d) 6720 e) 40,320 f) 3,628,800 7.15,120 9.1320 
11. a) 210 b) 386 c) 848 d) 252 13.2(n!)? 15. 65,780 
17. 2100—5051 19. a) 1024b) 45c) 176 d) 252 21. a) 120 
b) 24c) 120d) 24e)6f)0 23.609,638,400 25. a) 17,280 
b) 14,400 27. a) 94,109,400 b) 941,094 ce) 3,764,376 
d) 90,345,024 e) 114,072 f) 2328 g)24 _ h) 79,727,040 
i) 3,764,376 j) 109,440 29. a) 12,650 b) 303,600 
31. a) 37,927 b) 18,915 33. a) 122,523,030 b) 72,930,375 
c) 223,149,655 d) 100,626,625 35.54,600 37.45 
39. ‘_ 45.13 47.873 


(n—r)! 
Section 6.4 
1. x4 +40 y + 6x*y? + 4a ty*t 3.294 Gy + 15xty? + 


20x3y3+ 15x°y4+6xy+y% 5. 101 7. -20 (3) = -94,595,072 
200 z 
paner ALL HOA Ay = 2a" = 


810x!y3 + 1080x!2y6 — 720x8y? + 240x4y!? — 32y!5 
13. a) 71,680 b)O c)—16,384 d)-—35,840 e) —1,792 
15. (—1)200-0/3 (ge if k = 2 (mod 3) and —100 < k < 
200; O otherwise 17.1 9 36 84 126 126 84 36 
9 1 19. The sum ofall the positive numbers (hs as k runs 


from 0 to n, is 2”, so each one of them is no bigger than this 


n\ — n(n—l)\(n—2)--(n—k+1) MeN k pk] 
sum. 21. ( ) F112) < ie n* [2 
n ny _ n! n! = n! 
23. (ea) t (i) p TRAE + aw Kn)! k\(n—k+1)! [k+ 
= n+)! _ (n+ 
(n-k+1) = ETAN ( H ) 25. a) We show that each 


side counts the number of ways to choose from a set with n ele- 
ments a subset with k elements and a distinguished element of 
that set. For the left-hand side, first choose the k-set (this can 
be done in (7) ways) and then choose one of the k elements in 
this subset to be the distinguished element (this can be done in 
k ways). For the right-hand side, first choose the distinguished 
element out of the entire n-set (this can be done in n ways), 
and then choose the remaining k — 1 elements of the subset 
from the remaining n — 1 elements of the set (this can be done 


: n-1 (nm) _ 27, __n! _ n(n—1)! _ n-1 

in (0 wa r KC =k Ta = PDD = n(") 
n+ e n+1)! na n+ n! = 

27. C ) T ka- k (k-1)![n-(k-1)]! (+ 1) 

( a) j k. This identity together with (o) = | gives a recursive 
sys 2n 2n oe 2n+1\ _ 1 2n+1 

definition. 29. (G ) + os) = Cy) = 5 [ ne ) + 


ey] = [Pay TE E 
counts the number of ways to choose a sequence of r Os and 
n+ 1 1s by choosing the positions of the Os. Alternately, sup- 
pose that the (j + 1)st term is the last term equal to 1, so that 
n < j < n +r. Once we have determined where the last 
1 is, we decide where the Os are to be placed in the j spaces 
before the last 1. There are n 1s and j — n Os in this range. By 
the sum rule it follows that there are D hs ) = Dio (5) 
ways to do this. b) Let P(r) be the statement to be proved. The 
basis step is the equation C) = (a which is just 1 = 1. 
Assume that P(r) is true. Then Ar (E9) = Vico (ea) + 
CET) = (==) + CC = cre), using the induc- 
tive hypothesis and Pascal’s identity. 33. We can choose the 
leader first in n different ways. We can then choose the rest of 
the committee in 2”7! ways. Hence, there are 2”! ways to 
choose the committee and its leader. Meanwhile, the number 
of ways to select a committee with k people is G J: Once we 
have chosen a committee with k people, there are k ways to 
choose its leader. Hence, there are Jei k(") ways to choose 

k) = nt. 
35. Let the set have n elements. From Corollary 2 we have 
O= G+ G) - + CD) = 0. It follows that 
(+ G+) + = (1) + (4) + (2) +=. The left-hand side 
gives the number of subsets with an even number of elements, 
and the right-hand side gives the number of subsets with an 
odd number of elements. 37. a) A path of the desired type 
consists of m moves to the right and n moves up. Each such 
path can be represented by a bit string of length m + n with 
m Os and n 1s, where a 0 represents a move to the right and a 
1 a move up. b) The number of bit strings of length m + n 


the committee and its leader. Hence, a 


containing exactly n 1s equals (=) = (">") because such 
a string is determined by specifying the positions of the n 1s 
or by specifying the positions of the m Os. 39. By Exercise 
37 the number of paths of length n of the type described in 
that exercise equals 2”, the number of bit strings of length n. 
On the other hand, a path of length n of the type described in 
Exercise 37 must end at a point that has n as the sum of its co- 
ordinates, say (n — k, k) for some k between 0 and n, inclusive. 
By Exercise 37, the number of such paths ending at (n — k, k) 
equals (7) = (ee Hence, Xo (") = 2", 41. By Ex- 
ercise 37 the number of paths from (0, 0) to (n + 1, r) of the 
type described in that exercise equals ale ). But such a path 
starts by going j steps vertically for some j with 0 < j < r. The 
number of these paths beginning with j vertical steps equals 
the number of paths of the type described in Exercise 37 that 
go from (1, j) to (n + 1, r). This is the same as the number 
of such paths that go from (0, 0) to (n, r — j), which by Ex- 
ercise 37 equals en ). Because Yio ay = Dio (Y) 
it follows that X; _; (5) = (ai A 43. a) ("3") b) (E) 


c) one d) (nv2p) e) Largest odd entry in nth row of 


Pascal’s triangle f) (2) 


Section 6.5 


1.243 3.266 5.125 7.35 9.a) 1716 b) 50,388 
c) 2,629,575 d)330 11.9 13.4,504,501 15. a) 10,626 
b) 1,365 ¢) 11,649 d) 106 17.2,520 19. 302,702,400 
21. a) 169 b) 156 ¢)78 d)91 23.3003 25. 7,484,400 
27. 30,492 29. C(59, 50) 31.35 33. 83,160 35. 63 
37. 19,635 39. 210 41. 27,720 43. 52!/(7P 17!) 
45. Approximately 6.5 x 10°? 47.a)C(k + n- 1, n) 
b) (k + n — 1)!/(k — 1)! 49. There are C(n,n,) ways to 
choose n, objects for the first box. Once these objects are 
chosen, there are C(n — n, ny) ways to choose objects for the 
second box. Similarly, there are C(n — n; — m, nz) ways to 
choose objects for the third box. Continue in this way until 


there is C(n — ny — m yp) = Cy n) = 
1 way to choose the objects for the last box (because 
ny + Ny + ++ +n, = n). By the product rule, the num- 


ber of ways to make the entire assignment is C(n, n,)C(n — 
Ny, m)C(n — Ny — m, nz) = C(n — Ny — Ny — + — Ng- ng), 
which equals n!/(n,!n,! =- n;,!), as straightforward simpli- 
fication shows. 51. a) Because x; <x) < = <x,, it fol- 
lows that x +0 < x», +1 < < ob r= 1, 
The inequalities are strict because x, +j — 1 < Xy +j 


j 
as long as X S Xy Because 1 SxjSntr- 1, this se- 


quence is made up of r distinct elements from T. b) Suppose 
that 1 < x, < x < «+ < x, < n+r-— 1. Let 
Yk = Xy — (k — 1). Then it is not hard to see that y, < y,,, for 
k = 1,2,...,r-—1andthatl < y, < n fork = 1,2, ...r. 
It follows that {y,, Yz ..., y,} is an r-combination with rep- 
etitions allowed of S$. c) From parts (a) and (b) it follows 
that there is a one-to-one correspondence of r-combinations 
with repetitions allowed of § and r-combinations of T, a 
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set with n + r — 1 elements. We conclude that there are 
C(n +r — 1, r) r-combinations with repetitions allowed of S. 
53.65 55.65 57.2 59.3  61.a) 150 b) 25 c)6 d)2 
63. 90,720 65. The terms in the expansion are of the form 


ny n n . 
1X Xm", Where n) +n) + +++ + np = n. Such a term arises 


1:72 m 
from choosing the x, inn, factors, the x, in n, factors, ..., and 
the x,, in n,, factors. This can be done in C(n; n, No, ..., Nm) 
ways, because a choice is a permutation of n; labels “1,” n, 
labels “2,” ..., and n, labels “m.” 67. 2520 
Section 6.6 


1. 14532, 15432, 21345, 23451, 23514, 31452, 31542, 43521, 
45213, 45321 3. AAAI, AAA2, AAB1, AAB2, AAC], 
AAC2, ABAI, ABA2, ABB1, ABB2, ABC1, ABC2, ACA1, 
ACA2, ACB1, ACB2, ACC1, ACC2, BAA1, BAA2, BABI, 
BAB2, BACI, BAC2, BBA1, BBA2, BBB1, BBB2, BBC1, 
BBC2, BCAI, BCA2, BCB1, BCB2, BCC1, BCC2, CAA1, 
CAA2, CAB1, CAB2, CAC1, CAC2, CBA1, CBA2, CBB1, 
CBB2, CBC1, CBC2, CCA1, CCA2, CCB1, CCB2, CCC1, 
CCC2 5. a) 2134 b) 54132 c) 12534 d) 45312 7.1234, 
1243, 1324, 1342, 1423, 1432, 2134, 2143, 2314, 2341, 2413, 
2431, 3124, 3142, 3214, 3241, 3412, 3421, 4123, 4132, 4213, 
4231, 4312, 4321 9. {1,2,3}, {1, 2, 4}, {1, 2, 5}, {13,4}, 
{1, 3, 5}, {1 4 5}, (2, 3, 4}, {2, 3, 5}, {24 5}, (3, 4 5} 
11. The bit string representing the next larger r-combination 
must differ from the bit string representing the original one in 
position 7 because positions i + 1, ..., r are occupied by the 
largest possible numbers. Also a; + 1 is the smallest possi- 
ble number we can put in position i if we want a combination 
greater than the original one. Then a;+2, ...,a;+r—i+1 are 
the smallest allowable numbers for positions i + 1 to r. Thus, 
we have produced the next r-combination. 13. 123, 132, 
213, 231, 312, 321, 124, 142, 214, 241, 412, 421, 125, 152, 
215, 251, 512, 521, 134, 143, 314, 341, 413, 431, 135, 153, 
315, 351, 513, 531, 145, 154, 415, 451, 514, 541, 234, 243, 
324, 342, 423, 432, 235, 253, 325, 352, 523, 532, 245, 254, 
425, 452, 524, 542, 345, 354, 435, 453, 534, 543 15. We 
will show that it is a bijection by showing that it has an in- 
verse. Given a positive integer less than n!, let a), dy, ...,d,_1 
be its Cantor digits. Put n in position n—a,_,;thena,_, is the 
number of integers less than n that follow n in the permuta- 
tion. Then put n — 1 in free position (n — 1) — a,_5, where we 
have numbered the free positions 1, 2, ..., n — 1 (excluding 
the position that n is already in). Continue until 1 is placed 
in the only free position left. Because we have constructed an 
inverse, the correspondence is a bijection. 
17. procedure Cantor permutation(n, i: integers with 
n>land0<i<n)!) 
x:=n 
forj := 1 ton 
p;:=0 
for k := l ton- 1 
c := |x/(n- k)!];x:=x-—ce(n-k)!;h:=n 
while p, # 0 
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h:=h-1 
for j := 1 toc 
h:=h-1 
while p, 4 0 
h:=h-1 
P,i=n-k+1 
h:=1 
while p, # 0 
h:=h+1 
P := 1 
{P|P> --- Pn, is the permutation corresponding to i} 


Supplementary Exercises 


1. a) 151,200 b) 1,000,000 ¢)210 dq)5005 3.3! 
5. 24,600 7.a)4060 b)2688 c) 25,009,600 9. a) 192 
b) 301 c¢)300 d)300 11.639 13. The maximum pos- 
sible sum is 240, and the minimum possible sum is 15. So the 
number of possible sums is 226. Because there are 252 subsets 
with five elements of a set with 10 elements, by the pigeon- 
hole principle it follows that at least two have the same sum. 
15. a) 50 b) 50 c) 14 d) 17 17. Letay, ao, ..., a, be the in- 
tegers, and let d; = Ži aj. If d; = 0 (mod m) for some i, we 
are done. Otherwise d) mod m, d, mod 7n, ..., d,, mod m 
are m integers with values in {1, 2, ..., m — 1}. By the pi- 
geonhole principle d, = d, for some 1 < k < 1< m. 


Then Jl, a, = d,- d = 0 (modm). 19. The 


decimal erani of the rational number a/b can be ob- 
tained by division of b into a, where a is written with a 
decimal point and an arbitrarily long string of Os following 
it. The basic step is finding the next digit of the quotient, 
namely, |r/b], where r is the remainder with the next digit 
of the dividend brought down. The current remainder is ob- 
tained from the previous remainder by subtracting b times the 
previous digit of the quotient. Eventually the dividend has 
nothing but Os to bring down. Furthermore, there are only 
b possible remainders. Thus, at some point, by the pigeon- 
hole principle, we will have the same situation as had pre- 
viously arisen. From that point onward, the calculation must 
follow the same pattern. In particular, the quotient will re- 
peat. 21. a) 125,970 b) 20 c) 141,120,525 d) 141,120,505 
e) 177,100 f) 141,078,021 23.a)10 b)8 c)7 25.3" 
27.C(n + 2,r + 1)= C(n+1,r+ 1)+ Cn + 1,r)= 
2C(n + 1,r+1)- Cían+1l,r+1)+ Cín+1,r)= 
2C(n + l,r + 1) - (C(n,r + 1) + C(n, r)) + (C(n, r) + 
C(n,r— 1)) =2C(n+ l,r+1)-— C(n,r+ 1)+ C(n,r — 1) 
29. Substitute x = 1l andy = 3 into the binomial theo- 
rem. 31. Both sides count the number of ways to choose a 
subset of three distinct numbers {i, j, k} with i < j < k from 
{1,2,...,n}. 33. C(n+ 1,5) 35.3,491,888,400 37.5% 
39.a)45 b)57 ¢c)12 41.a)386 b)56 43.0ifn < m; 
C(n— l,n-m)ifn> m 45.a)15,625 b)202 c) 210 
d)10 47. a)3 b)11 c)6 d)10 49. There are two pos- 
sibilities: three people seated at one table with everyone else 
sitting alone, which can be done in 2C(n, 3) ways (choose the 


three people and seat them in one of two arrangements), or 
two groups of two people seated together with everyone else 
sitting alone, which can be done in 3C(n, 4) ways (choose four 
people and then choose one of the three ways to pair them 
up). Both 2C(n, 3) + 3C(n, 4) and (3n — 1)C(n, 3)/4 equal 
nt/8 — 5n? /12 + 3n?/8 —n/12. 51. The number of per- 
mutations of 2n objects of n different types, two of each type, 
is (2n)!/2”. Because this must be an integer, the denominator 
must divide the numerator. 53. CCGGUCCGAAAG 

55. procedure next permutation(n: positive integer, 


41, Ay, ..., a,: positive integers not exceeding 
n with aja ...a, #nn...n) 
[7 
while a; = n 
a :=1 
i:=i-1 
a;:=a;+1 
{aja ... a, is the next permutation in lexicographic 
order} 


57. We must show that if there are R(m, n — 1) + R(m — 1, n) 
people at a party, then there must be at least m mutual friends 
or mutual enemies. Consider one person; let’s call him Jerry. 
Then there are R(m— 1, n)+R(m, n—1)—1 other people at the 
party, and by the pigeonhole principle there must be at least 
R(m — 1, n) friends of Jerry or R(m, n — 1) enemies of Jerry 
among these people. First let’s suppose there are R(m — 1, n) 
friends of Jerry. By the definition of R, among these people 
we are guaranteed to find either m— 1 mutual friends or n mu- 
tual enemies. In the former case, these m — 1 mutual friends 
together with Jerry are a set of m mutual friends; and in the 
latter case, we have the desired set of n mutual enemies. The 
other situation is similar: Suppose there are R(m, n — 1) ene- 
mies of Jerry; we are guaranteed to find among them either m 
mutual friends or n — 1 mutual enemies. In the former case, 
we have the desired set of m mutual friends, and in the latter 
case, these n — 1 mutual enemies together with Jerry are a set 
of n mutual enemies. 


CHAPTER 7 


Section 7.1 


1. 1/13 3.1/2 5.1/2 7.1/64 9.47/52 11. 1/C(52, 5) 
13.1 — [C(48, 5)/C(52, 5)] 15. C(13, 2)C(4, 2)C(4, 2) 
C(44, 1)/C(52, 5) 17. 10,240/C(52, 5) 19. 1,302,540/ 
C(52, 5) 21.1/64 23.8/25 25.a)1/C(50, 6) = 
1/15,890,700 b) 1/C(52, 6) = 1/20,358,520 
c) 1/C(56, 6) = 1/32,468,436 d) 1/C(60, 6) = 1/50,063, 860 
27. a) 139, 128/319,865 b) 212,667/511,313 
c) 151,340/386,529 d) 163,647/446,276 29. 1/C(100, 8) 
31.3/100 33. a) 1/7,880,400 b) 1/8,000,000 35. a) 9/19 
b) 81/361 c)1/19 d) 1,889,568/2,476,099 e) 48/361 
37. Three dice 39. a) 4/756,438,375 b) 13/30,257,535 


c) 4888/2,750,685 d) 90,272/9,823,875 41. a) 25/ 
292,201,338 b) 25/292,201,338 c) 1280/571,120,797 
d) 7564/661,089 43. The door the contestant chooses is 
chosen at random without knowing where the prize is, but the 
door chosen by the host is not chosen at random, because he 
always avoids opening the door with the prize. This makes 
any argument based on symmetry invalid. 45. a) 671/1296 
b) 1 — 35%4/3674; noc) The former 


Section 7.2 


1. p(T) = 1/4, p(H) = 3/4 3. pl) = p6) = p65) 

p(6) = 1/16; p22) = p(4) = 3/8 5.9/49 7.a)1/2 
b) 1/2 ce) 1/3 d) 1/4 e) 1/4 9. a) 1/26! b) 1/26 ¢) 1/2 
d) 1/26 e) 1/650 f)1/15,600 11. Clearly, p(E U F) > 
P(E) = 0.7. Also, p(E U F) < 1. If we apply Theorem 2 from 
Section 7.1, we can rewrite this as p(E)+p(F)—p(ENF) < 1, 
or 0.7 + 0.5 — p(E N F) < 1. Solving for p(E N F) gives 
P(ENF) > 0.2. 13. Because p(EUF) = p(E)+p(F)—p(EnF) 
and p(E U F) < 1, it follows that 1 > p(E) + p(F) — pE A F). 
From this inequality we conclude that p(E) + p(F) < 1 + 
P(ENF). 15. We will use mathematical induction to prove 
that the inequality holds for n > 2. Let P(n) be the state- 
ment that PU, E) < 2i p(E;). Basis step: P(2) is true 
because p(E, U E,) = p(E;) + p(E,) — p(E; N E,) < 
P(E,) + p(E,). Inductive step: Assume that P(k) is true. Us- 
ing the basis case and the inductive hypothesis, it follows that 
PU E) < PUL, E) +pEa) < Lot pE). This 
shows that P(k + 1) is true, completing the proof by math- 
ematical induction. 17. Because E U E is the entire sam- 
ple space S, the event F can be split into two disjoint events: 
F = SAF = (EUE)NF = (ENF)U(ENF), using the distributive 
law. Therefore, p(F) = p((ENF)U(ENF)) = p(ENF)+p(EnF), 
because these two events are disjoint. Subtracting p(E N F) 
from both sides, using the fact that p(E N F) = p(E) - p(F) 
(the hypothesis that E and F are independent), and factoring, 
we have p(F)[1— p(E)] = p(EN F). Because 1 — p(E) = p(E), 
this says that ENF) = pE) -p(F), as desired. 19. a) 1/12 
b)i- 2.2.. 2 e)5 21.614 23.1/4 25.3/8 
27. a) Not independent b) Not independent c) Not inde- 
pendent 29.3/16 31.a)1/32 = 0.03125 b)0.49 x 
0.02825 e) 0.03795012 33. a) 5/8 b) 0.627649 c) 0.6431 
35.a p" bi - p op +n- p. ( = p) 
d)1—[p"+n-p™!-(1—p)] 37. p(U22, E) is the sum of 
p(s) for each outcome s in | JŽ, E;. Because the Ejs are pair- 
wise disjoint, this is the sum of the probabilities of all the out- 
comes in any of the E;s, which is what })*°, p(E;) is. (We can 
rearrange the summands and still get the same answer because 
this series converges absolutely.) 39. a) E = Uj ʻi F so the 
given inequality now follows from Boole’s inequality (Exer- 
cise 15). b) The probability that a particular player not in the 
jth set beats all k of the players in the jth set is (1/2) = 2%. 
Therefore, the probability that this player does not do so is 
1—2-*, so the probability that all m—k of the players not in the 
jth set are unable to boast of a perfect record against everyone 
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in the jth set is (1 -2-*)"-*. That is precisely pE je c) The first 
inequality follows immediately, because all the summands are 
the same and there are (") of them. If this probability is less 


than 1, then it must be possible that E fails, i.e., that E hap- 
pens. So there is a tournament that meets the conditions of the 
problem as long as the second inequality holds. d) m > 21 
for k = 2, and m > 91 fork = 3 
41. procedure probabilistic prime(n, k) 
composite := false 
i:=0 
while composite = false and i < k 
i:=i+1 
choose b uniformly at random with 1 < b < n 
apply Miller’s test to base b 
if n fails the test then composite := true 
if composite = true then print (“composite”) 
else print (“probably prime”) 


Section 7.3 


NOTE: In the answers for Section 7.3, all probabili- 
ties given in decimal form are rounded to three decimal 
places. 1.3/5 3.3/4 5.0.481 7.a)0.999 b)0.324 
9. a) 0.740 b) 0.260 c)0.002 d)0.998 11. 0.724 
13.3/17 15.a)1/3 b)p(.M =j| W =k = lifi, j, 
and k are distinct; pM = j | W = k) = O0ifj = korj = i; 
DM =j|W = k) = 1/2ifi = kandj#i ¢)2/3 d)You 
should change doors, because you now have a 2/3 chance to 
win by switching. 17. The definition of conditional proba- 
bility tells us that p(F; | E) = p(E N F;) /p(E). For the numer- 
ator, again using the definition of conditional probability, we 
have p(E N F;) = p(E | F)p(F,), as desired. For the denom- 
inator, we show that p(E) = )7_, p(E | F;)p(F;). The events 
E N F; partition the event Æ; that is, (E N F, ) n (EN F;,) = Ø 
when i; # i, (because the F;’s are mutually exclusive), and 
Ui (E N F,,) = E (because the L_, F; = S). Therefore, 
PE) = Xia PEN F) = Yi, PE | Fpp). 19.No 
21. Yes 23. By Bayes’ theorem, p(S | E, NE,) = p(E NE, | 
S)p(S)/[p(E, NE, | S)p(S) + p(E, N E, | S)p(S)]. Because we 
are assuming no prior knowledge about whether a message is 
or is not spam, we set p(S) = pS) = 0.5, and so the equation 
above simplifies to p(S | E; NE) = p(E NE» | S)/[p(E, NE; | 
S)+p(E, NE, | S)]. Because of the assumed independence of 
E,, E,, and S, we have p(E, N E, | S) = p(£ | $) - p(E, | $), 
and similarly for S. 


Section 7.4 


1. 2.5 3.5/3 5.336/49 7.170 9.(4n + 6)/3 
11. 50,700,551/10,077,696 + 5.03 13.6 15. p(X > j)= 
EPX = 4) = Dep) 'p = pl -py! Yep) = 
pa- pya-d-p) = @- pi! 17.2302 
19. (7/2) - 7 # 329/12 21.10 23.1472 pounds 
25.p+(n—1)p(1-p) 27.5/2 29.a)0 b)n 31. This 
is not true. For example, let X be the number of heads in 
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one flip of a fair coin, and let Y be the number of heads in 
one flip of a second fair coin. Then A(X) + A(Y) = 1 but 
A(X + Y) = 0.5. 33. a) We are told that X, and X, are 
independent. To see that X, and X; are independent, we enu- 
merate the eight possibilities for (X,, X,, X;) and find that 
(0, 0, 0), (1, 0, 1), (0, 1, 1), (1, 1, 0) each have probability 1/4 
and the others have probability 0 (because of the definition 
of X;). Thus, p(X; = 0A X = 0) = 1/4, p(X, = 0) = 1/2, 
and p(X; = 0) = 1/2, so it is true that p(X, = 0A X, = 
0) = p(X, = 0)p(X% = 0). Essentially the same calculation 
shows that p(X, = 0A X; = 1) = p(X, = O)p(X3 = 1), 
pX = 1AX; 0) D(X, 1)p(X3 = 0), and 
P(X, = 1AX; = 1) = p(X, = 1)p(X; = 1). Therefore, by defi- 
nition, X, and X, are independent. The same reasoning shows 
that X, and X; are independent. To see that X} and X; +X; are 
not independent, we observe that p(X, = 1AX,+X, = 2) =0. 
But p(X, = DPX +X, = 2) = (1/2)(1/4) = 1/8. b) We 
see from the calculation in part (a) that X,, X,, and X, are 


all Bernoulli random variables, so the variance of each is 
(1/2)(1/2) = 1/4. Therefore, V(X,) + V(X,) + V(X) = 3/4. 
We use the calculations in part (a) to see that E(X, +X, +X3) = 
3/2, and then V(X, +X, +X3) = 3/4. c) In order to use the first 
part of Theorem 7 to show that V((X, +X) +- +X +X) = 
V(X, +X, + + + X) + V(X;+1) in the inductive step of a 
proof by mathematical induction, we would have to know 
that X; + X, + = + X, and X,,, are independent, but we 
see from part (a) that this is not necessarily true. 35. 1/100 
37. E(X)/a = Yi(r/a) + p(X = r) = pa 1-pX® = 
r) = p(X >a) 39.a) 10/11 b) 0.9999 41. a) Each of 
the n! permutations occurs with probability 1/n!, so E(X) is 
the number of comparisons, averaged over all these permu- 
tations. b) Even if the algorithm continues n — 1 rounds, X 
will be at most n(n — 1)/2. It follows from the formula for ex- 
pectation that E(X) < n(n — 1)/2. c) The algorithm proceeds 
by comparing adjacent elements and then swapping them if 
necessary. Thus, the only way that inverted elements can be- 
come uninverted is for them to be compared and swapped. 
d) Because X(P) > I(P) for all P, it follows from the defi- 
nition of expectation that E(X) > EU). e) This summation 
counts 1 for every instance of an inversion. f) This follows 
from Theorem 3. g)By Theorem 2 with n = 1, the ex- 
pectation of J;, is the probability that a, precedes a; in the 
permutation. This is clearly 1/2 by symmetry. h) The sum- 
mation in part (f) consists of C(n, 2) = n(n — 1)/2 terms, 
each equal to 1/2, so the sum is n(n — 1)/4. i) From part 
(a) and part (b) we know that E(X), the object of interest, 
is at most n(n — 1)/2, and from part (d) and part (h) we 
know that E(X) is at least n(n — 1)/4, both of which are 
@(n?). 43.1 45. V(X +Y) = E(X + Y) - E(X + Y? = 
E(X?+2XY+Y7)-[E(X)+E(Y)[ = E(X?)4+2E(XY)+E(¥7)— 
E(X) — 2E(X)E(Y) — E(Y)? = E(X?) — EXP + 2[E(XY) - 
E(X)E(Y)] + EY?) — EY)? = V(X) + 2 Cov(X, Y) + V(Y) 
47. [(n — 1)/n]” 49. (n — 1)” /n"! 


Supplementary Exercises 


1. 1/109,668 3. a)1/195,249,054 b) 1/5,138,133 
c) 45/357,599 d) 18,285/18,821 5. a)1/C(52, 13) 
b) 4/C(52, 13) œ) 2,944,656/C(52, 13) d) 35,335,872/ 
C(52, 13) 7.a)9/2 b)21/4 9.a)9 b)21/2 1l.a)8 
b) 49/6 13. a)n/2""'_b) pd — p)!, where p = n/2"7! 
c) 2" 1/n 15, Sect t 17,a)2/3 b) 2/3 
19. 1/32 21. a) The probability that one wins 2” dollars 
is 1/2”, because that happens precisely when the player gets 
n— l tails followed by a head. The expected value of the win- 
nings is therefore the sum of 2” times 1/2” as n goes from 1 to 
infinity. Because each of these terms is 1, the sum is infinite. 
In other words, one should be willing to wager any amount of 
money and expect to come out ahead in the long run. b) $9, 
$9 23.a) 1/3 when S = {1, 2, 3, 4,5, 6, 7, 8, 9, 10, 11, 12}, 


A = {1,2,3,4 5,6,7,8,9}, and B = {I, 2, 3, 4}; 
1/12 when S = {1, 2, 3, 4, 5,6 7, 8, 9, 10, 11, 12}, 
A = {4,5,6, 7,8,9, 10, 11, 12}, and B = {1, 2,3, 4} 
b)1 when S = (1, 2,3, 4,5, 6,7, 8, 9, 10, 11, 12}, 
A = {4,5,6,7, 8,9, 10, 11, 12}, and B = {1, 2,3, 4}; 
3/4 when S = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}, 
A = {1,2,3,4 5,6,7,8,9}, and B = {1,2,3,4} 
25.a)p(E, n E) = pEpDpE), pE n B) = 


P(E, )p(E3), pE N E3) = p(E)p(E3), pE NE, NE) = pE) 
DP(E,)p(E;) b) Yes c) Yes; yes d) Yes; no e)2”-—n-1 
27. a) 1/2 under first interpretation; 1/3 under second in- 
terpretation b) Let M be the event that both of Mr. Smith’s 
children are boys and let B be the event that Mr. Smith chose 
a boy for today’s walk. Then p(M) = 1/4, p(B | M) = 1, 
and p(B | M) = 1/3. Apply Bayes’ theorem to compute 
p(M | B) = 1/2. c) This variation is equivalent to the 
second interpretation discussed in part (a), so the answer is 
unambiguously 1/3. 29. V(aX + b) = E((aX + by) - 
E(aX + b? = E(a?X? + 2abX + b°) — [aE(X) + bP 
E(a?X*) + E(2abX) + E(b*) — [a E(X} + 2abE(X) + b*] = 
@ E(X?) + 2abE(X) + b? — @ E(X? — 2abE(X) — b»? = 
a [E(X*) — E(X] = a?V(X) 31. To count every el- 
ement in the sample space exactly once, we must include 
every element in each of the sets and then take away the 
double counting of the elements in the intersections. Thus, 
pe, UE,U++-VE,,) = P(E, )+p(Ey) ++ +p En) -pE NE3)— 
p(E NE3)—+:—p(E, NE, )—p(E,NE3)—p(E,NE4)— +: —p(E N 
Em pE n- NEn) = gm—(m(m—1)/2)r, because C(m, 2) 
terms are being subtracted. But p(E, UE, U + U Em) = 1, so 
we have gm—[m(m-—1)/2]r = 1. Because r > 0, this equation 
tells us that gm > 1, so q > 1/m. Because q < 1, this equation 
also implies that [m(m-—1)/2]r = qm— 1 < m- 1, from which 
it follows that r < 2/m. 33. a) We purchase the cards until 
we have gotten one of each type. That means we have pur- 
chased X cards in all. On the other hand, that also means that 
we purchased X cards until we got the first type we got, and 
then purchased X, more cards until we got the second type we 
got, and so on. Thus, X is the sum of the X;s. b) Once j distinct 
types have been obtained, there are n — j new types available 
out of a total of n types available. Because it is equally likely 


ll 


that we get each type, the probability of success on the next 
purchase (getting a new type) is (n — j)/n. e) This follows 
immediately from the definition of geometric distribution, 
the definition of X T and part (b). d) From part (c) it follows 
that E(X;) = n/(n — j). Thus, by the linearity of expectation 
and part (a), we have E(X) = E(Xo) + E(X,) + +++ + E(X„—1) 


att peg tan(t+ teat), e) About 


n a~l 


CHAPTER 8 


Section 8.1 


1. Let P(n) be “H, = 2” — 1.” Basis step: P(1) is true because 
H, = 1. Inductive step: Assume that H, = 2” — 1. Then be- 
cause H,,,, = 2H, + 1, it follows that H,,,, = 2(2”-1)+1= 
2+1 _ 1, 3.a)a, = 2a,_, +4,-5 forn > 5 b)a = 1, 
a 2, dy 4, az 8, a4 16 ¢)1217 5.9494 
7.a) dy = pj +4,2 +2" forn > 2 b) ag =0, a, =0 ©) 94 
9. a) a, = a,-1 +4,2 +4,3 forn > 3 b) dy = 1,a, =2, a, = 4 
c) 81 1ll.a)a, =a, +4,2 forn>2 b)ay=1a,=1 
c¢)34 13.a)a„ =2a,_,+2a,_, forn > 2 b) ay = 1,a, =3 
c) 448 «15. a) a, =2a,_,+a,_, forn > 2 b) ay) = 1,a, =3 
c) 239 17.a)a, =2a,_, frn > 2 bya, = 3 ©) 96 
19.a)a, = a,; +4,» forn > 2 b)a = lia, = 1 
©) 89 2la)R, =n+R,_,,Ryo =1 b) R, =n(n+1)/2+1 
23. a) S, =S,,_,+(n?-n+2)/2, Sy = 1 b) S, = (n? +5n+6)/6 
25.64 27.a) a, =2a,_;+2a,_, b) dg = 1, a, = 3 c) 1224 
29. Clearly, S(m, 1) = 1 for m > 1. If m > n, then a function 
that is not onto from the set with m elements to the set with 
n elements can be specified by picking the size of the range, 
which is an integer between 1 and n — 1 inclusive, picking the 
elements of the range, which can be done in C(n, k) ways, and 
picking an onto function onto the range, which can be done 
in S(m, k) ways. Hence, there are Ei C(n, k)S(m, k) func- 
tions that are not onto. But there are n” functions altogether, 
so S(m, n) = n” — "| Cin, K)S(m, k). 31. a) C; = CoC4 + 
C,C34+C,C,+C3C,+C4Cp = 1:14+1-5+2-2+5-1+14-1 = 42 
b) C(10, 5)/6 = 42 33.J(1) = 1, J(2) = 1, J(3) = 3, 
J(4) = 1, J(5) = 3, J(6) = 5, J(T) = 7, J(8) = 1, J(9) = 3, 
J(10) = 5, J(11) = 7, J(12) = 9, J(13) = 11, J14 = 13, 
J(15) = 15, J(16) = 1 35. First, suppose that the num- 
ber of people is even, say 2n. After going around the circle 
once and returning to the first person, because the people at 
locations with even numbers have been eliminated, there are 
exactly n people left and the person currently at location i is 
the person who was originally at location 2i — 1. Therefore, 
the survivor [originally in location J(2n)] is now in location 
J(n); this was the person who was at location 2J (n)— 1. Hence, 
J(2n) = 2J(n) — 1. Similarly, when there are an odd number 
of people, say 2n + 1, then after going around the circle once 
and then eliminating person 1, there are n people left and the 
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person currently at location 7 is the person who was at location 
2i+ 1. Therefore, the survivor will be the player currently oc- 
cupying location J(n), namely, the person who was originally 
at location 2/(n) + 1. Hence, J(2n + 1) = 2J(n) + 1. The 
basis step is J(1) = 1. 37. 73,977, 3617 39. These nine 
moves solve the puzzle: Move disk 1 from peg 1 to peg 2; 
move disk 2 from peg | to peg 3; move disk 1 from peg 2 to 
peg 3; move disk 3 from peg | to peg 2; move disk 4 from 
peg 1 to peg 4; move disk 3 from peg 2 to peg 4; move disk 1 
from peg 3 to peg 2; move disk 2 from peg 3 to peg 4; move 
disk 1 from peg 2 to peg 4. To see that at least nine moves 
are required, first note that at least seven moves are required 
no matter how many pegs are present: three to unstack the 
disks, one to move the largest disk 4, and three more moves 
to restack them. At least two other moves are needed, because 
to move disk 4 from peg | to peg 4 the other three disks must 
be on pegs 2 and 3, so at least one move is needed to restack 
them and one move to unstack them. 41. The base cases are 
obvious. If n > 1, the algorithm consists of three stages. In 
the first stage, by the inductive hypothesis, R(n — k) moves 
are used to transfer the smallest n — k disks to peg 2. Then 
using the usual three-peg Tower of Hanoi algorithm, it takes 
2* — 1 moves to transfer the rest of the disks (the largest 
k disks) to peg 4, avoiding peg 2. Then again by the in- 
ductive hypothesis, it takes R(n — k) moves to transfer 
the smallest n — k disks to peg 4; all the pegs are available 
for this, because the largest disks, now on peg 4, do not inter- 
fere. This establishes the recurrence relation. 43. First note 
that R(n) = pa [R(j) — R(j — 1)] [which follows because the 
sum is telescoping and R(0) = 0]. By Exercise 42, this is the 
sum of 2'’-! for this range of values of j. Therefore, the sum 
is EL i2'-!, except that if n is not a triangular number, then 
the last few values when i = k are missing, and that is what 
the final term in the given expression accounts for. 45. By 
Exercise 43, R(n) is no larger than EL i2'-|, It can be shown 
that this sum equals (k + 1)2* — 2%+! + 1, so it is no greater 
than (k + 1)2*. Because n > k(k— 1)/2, the quadratic formula 
can be used to show that k < 1+ /2n for alln > 1. Therefore, 
R(n) is bounded above by (1 + V2n + 1)2!+V2" < 8y/n2V2" 


for alln > 2. Hence, R(n) is O(y/n2Y™). 47.a)0 b)0 
c)2 d)2"!1-2"? 49.a, —2Va, + Va, = a, — 2(a, — 


a,1) + (Va, — Va,_1) = —a, + 2a,_, + ia, — 4,1) - 
(CA ore a„-2)] = a, + 2a, + (4, ~ 2a,_| + an2) = ay_2 
51. a, = a, + 4,3 = (a, — Va,) + (a, — 2Va, + V7a,) = 


2a, — 3Va, + Va,, ora, = 3Va, — Va, 53. Insert 
S(O) := Ø after T(0) := 0 (where S(j) will record the optimal 
set of talks among the first j talks), and replace the statement 
T(j) := max(w; + TEU), TG — 1)) with the following code: 
if w; + T(p(y)) > TU — 1) then 
T(j) := w; + TOG) 
SG) := SPO) U {j} 


else 
TG) :=TG-1) 
SG) = SG — 1) 


55. a) Talks 1, 3, and 7 b) Talks 1 and 6, or talks 1, 3, 
and 7 c) Talks 1,3, and 7 d) Talks 1 and6 57. a) This 
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follows immediately from Example 5 and Exercise 43c in 
Section 8.4. b) The last step in computing Aj; is to mul- 
tiply A; by Azı; for some k between i and j — 1 in- 
clusive, which will require mjm,,,mj,, integer multiplica- 
tions, independent of the manner in which A, and Apy; 
are computed. Therefore, to minimize the total number of 
integer multiplications, each of those two factors must be 
computed in the most efficient manner. c) This follows 
immediately from part (b) and the definition of M(i, j). 
d) procedure matrix order(m,, ..., Mp41: 
positive integers) 
for i:= 1 ton 
MiG, i) := 0 
for d:=1ton-1 
fori := 1 ton- d 
min := 0 
fork :=itoi+d 
new := M(i, k) + M(k + 1, i + d) + mmy} 1Mizd+1 
if new < min then 
min := new 
where(i, i + d) := k 
M(i, i+ d) := min 


e) The algorithm has three nested loops, each of which is in- 
dexed over at most n values. 


Section 8.2 


1. a) Degree 3 b) No c) Degree 4 d) No e) No f) Degree 2 
g) No 3.a)a, = 3-2" bja, = 2 oa = 
3-2"-2-3" dja, = 6-2"-2-n2" e)a, = n(-2)"! 
Ha, = 2" — (-2)" ga, = (1/2)! — (-1/2)"! 
n+1 n+1 
gia (44%) -5 (=$) 7. [2+ + (-1)"1/3 
9.a)P, = 1.2P,_, + 0.45P„,-2 Po = 100,000, P) = 
120,000 b) P,, = (250,000/3)(3/2)” + (50,000/3)(—3/10)" 
11. a) Basis step: For n = 1 we have 1 = 0 + 1, and for n = 2 
we have 3 = 1 + 2. Inductive step: Assume true for k < n. 
Then Lati = La + Ly = Ja- + frst + fir ETa = Gia + 


Ina) + Uns th) =h thao BL, = (BE) + (44) 

13. a,, = 8(—1)” — 3(-2)" +4-3” 15.4, =5+3(—2)" — 3" 
17.Leta, = C(n, 0) + Ca — 1,1) + = + Ca — k, k) 
where k = |n/2]. First, assume that n is even, so that 
k = n/2, and the last term is C(k, k). By Pascal’s identity we 
have a, = 1 + C(n — 2, 0) + C(n — 2, 1) + C(n - 3, 1) + 
C(n-— 3,2) + = + C(n—k,k-2)+ C(n-k k-1)+1 = 
1+C(n—2, 1)+C(n—3,2)+---+C(n—k,k—1)+C(n—2, 0)+ 
C(n— 3, 1) + +++ Cn- k k—-2)+ 1 = a, +a, because 
l(a — 1)/2] = k— 1 = |(n — 2)/2]. A similar calculation 
works when n is odd. Hence, {a,,} satisfies the recurrence re- 
lation a„ = d,_, +4,_ for all positive integers n, n > 2. Also, 
a, = C(1, 0) = 1 and a, = C(2, 0) + C(1, 1) = 2, which are 
fa and fz. It follows that a, = f,,,, for all positive integers n. 
19.a, = (n? +3n+5)-1)" 21. (ayy + ayn + a, yn? + 


Ay 31°) +(dy 9 +4y | N+Ay .N”)(—2)" +a; 9 +43, :1)3" +a; 9(—4)" 
23. a) 3a,_; +2" = 3(-2)" + 2” = 2"(-3 + 1) =-2" =a, 
b)a, = a3" — 2"! eja, = 371-2"! 25.a)A = 
—1,B = -7 b)a, =a2"-n—-7T o)a, = 11-2"-n-7 
27. a) psn? +p yn? +pin+po b) n’py(—2)" €) n*(pjntpo)2” 
d) (pn? + pin + po)4” e)n?(pan? + pin + po)(—2)" 
f) n°(pyn* + pa? + pon? + pint po)2"” g)py 29. a)a, = 
a2” + 3"+ bja, = -2- 2" + 3%! Bla, = 
a2" +3" —n-2"*!43n/2421/4 33. a, = (a+ pn+n?° +n? /6)2” 
35. a, = —4 - 2” — n? /4 —5n/2+ 1/8 + (39/8)3" 37. a, = 
n(n+1)(n+2)/6 39.a)1,—1,i, —i b)a, = ; = HOD" + 
timg Hi" 41. a) Using the formula for f,, we see that 


k- a (5) 5 (=$) < 1/v5 < 1/2 This 
means that f, is the integer closest to (= . b) Less 
when n is even; greater whennisodd 43. a„ =f, _,+2f,-1 
45.a)a, = 3a,_, + 4a,.,49 = 2,4, = 6 b)a, = 
[4-42 + (—1)]/5 47.a)a, = 2a,,, + (n — 1)10,000 
b)a, = 70,000 - 2"! — 10,000n — 10,000 49.a, = 
5n?/12 + 13n/12 + 1 51. See Chapter 11, Section 5 in 


[Ma93]. 53. 6" - 4"! /n 


Section 8.3 


1. 14 3.The first step is (1110) (1010) = (24 + 
20D, (10). + PAD — (10),][(10), — (10)] + 
(27 + 1)(10), (10). The product is (10001100),. 
5.C = 50, 665C + 729 = 33,979 7.a)2 b)4 
c)7 9. a)79 b) 48,829 c) 30,517,579 11. O(log n) 
13. O(n'®3?) 15.5 17. a) Basis step: If the sequence has 
just one element, then the one person on the list is the winner. 
Recursive step: Divide the list into two parts—the first half 
and the second half—as equally as possible. Apply the algo- 
rithm recursively to each half to come up with at most two 
names. Then run through the entire list to count the number 
of occurrences of each of those names to decide which, if ei- 
ther, is the winner. b) O(mlogn) 19. a) f(n) = f(n/2) +2 
b) Odogn) 21.a)7 b) O(logn) 
23. a) procedure largest sum(dy, ..., dy, 
best := 0 {empty subsequence has sum 0} 
for i := 1 ton 
sum := 0 
for j:=i+l1ton 
sum := sum + dj 
if sum > best then best := sum 
{best is the maximum possible sum of numbers 
in the list} 
b) O(n?) c) We divide the list into a first half and a second 
half and apply the algorithm recursively to find the largest 
sum of consecutive terms for each half. The largest sum of 
consecutive terms in the entire sequence is either one of these 
two numbers or the sum of a sequence of consecutive terms 
that crosses the middle of the list. To find the largest possi- 
ble sum of a sequence of consecutive terms that crosses the 
middle of the list, we start at the middle and move forward 
to find the largest possible sum in the second half of the list, 


and move backward to find the largest possible sum in the 
first half of the list; the desired sum is the sum of these two 
quantities. The final answer is then the largest of this sum 
and the two answers obtained recursively. The base case is 
that the largest sum of a sequence of one term is the larger 
of that number and 0. d) 11,9, 14 e) S(m) = 2S(n/2) +n, 
C(n) = 2C(n/2)+n+2, S(1) = 0, C(1) = 1 f) O(log n), bet- 
ter than O(n?) 25. (1, 6) and (3, 6) at distance 2 27. The 
algorithm is essentially the same as the algorithm given in 
Example 12. The central strip still has width 2d but we need 
to consider just two boxes of size d X d rather than eight 
boxes of size (d/2) x (d/2). The recurrence relation is the 
same as the recurrence relation in Example 12, except that 
the coefficient 7 is replaced by 1. 29. With k = log, n, it 
follows that f(n) = a‘f(1) + D: pde(n/b)! = = af(1) + 
D- = cn? = akf(l) + ken? = al%"f(1) + c(log, nn = 

n!S4F(1) + cnf log,n = n@f(1) + cn? log, n. 31. Let 
k = log,n where n is a power of b. Basis step: Ifn = 1 
and k = 0, then cun? + cn% = cy +c, = bfc/ 
(b! — a) + f) + bfc/(a — b*) = f(A). Inductive step: 
Assume true for k, where n = bt. Then for n = b‘+!, f(n) = 
af(n/b) + cn? = af[b%c/(b4 — a)\(n/b)4 + FA) + bfc/ 
(a — b*)] - (n/by%)} + cn? = bte/(bt — anta/b4 + 
FA) + btc/(a — b*)\n°&4 + cn? = nfjac/(b! — a) + 
c(bt — a)/(b4 — a)] + [fA) + b%e/(a — bfo] = 
[b¢c/(bt — an? + FA) + btc/(a — bY) \n'%4, 33. If 
a > bf, then log, a > d, so the second term dominates, 
giving O(n'&7). 35. O(n!°849) 37. O(m?) 


Section 8.4 


1. fœ) = 20°-1)/@-1) 3.a) f(x) = 2x0 — x®)/ 

x) b) P/U —x) 09x/0 —x) d)2/(1 - 2x) 9 (+ 
f) 2/(+x) 9 1/0-9]-2 h)x8/(1-x? 5.a)5/(1-x) 
b) 1/4 —3x) ©) 2x7/U—x) d) BG-»/ (1—-x)? © (+x 
7. a) dy = —64, a, = 144, ay = —108, a3 = 27, anda, = 0 
for alln > 4 b) The only nonzero coefficients are ay = 1, 
a; = 3, = 3,aọ = 1. c)a, = 5 da, = (3) for 
n > 3,and day =a, = 4, = 0 e)a = 8,a, = 3,a = 2, 
a, = 0 for odd n greater than 2 and a,, = 1 for even n greater 
than2 f)a, = 1ifn is a positive multiple 4, a, = —1 if 
n < 4, anda, = 0 otherwise g)a, = n— l forn > 2 
anda, = a, = 0 h)a, = 2"+!/n! 9.a)6 b)3 œ©)9 
d)O e)5 11.a) 1024 b) 11 c) 66 d)292,864 e) 20,412 
13.10 15.50 17.20 19.f@%) = 1/[d - ə» — x) 
a -xÑ —x!%)] 21.15 23.apx4 +x 42° + )/ 
(1—x) b) 6 25. a) The coefficient of x” in the power series 
expansion of 1 /[(1—»°)(1—x*)(1—x?°)] b) 1/0 -x° —x4—x?°) 
c)7 d) 3224 27. a) The generating function is (1 +x +x? + 
4 xt\(L tx ta) HEHA pot Ob patti +204 + 
PH 4x 4) = Bt x47 4 4x0 4x42°)/[0 -— 
x°)(1—x)(1—29)] = 0 4344.79 + 1209 + 194742728 43709 + 
48x! +61x!! +75x!? + ---. The coefficient of x” is the answer. 
b)75 29.a)3 b) 29 c)29 d) 242 3l.a) 10 b) 49 c)2 
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d)4 33.a) GX) — dy — ax — yx" b) GX’) 9 x*G() 
d) GX e) f “Gd )G@/-x) 35.aq,=2-3*-1 


37. a, = 18-3*-12-2" 39. a, = k? + 8k +20 + (6k — 18)2* 
41. Let Gx) = Dey fix". After shifting indices of summa- 
tion and adding series, we see that G(x) — xG(x) — x? G(x) = 
fot -x+ Deo fe- -fe = Ot e+ YP, Ot. 
Hence, G(x) — xG(x) — xX G(x) = x. Solving for G(x) gives 
G(x) = x/(1—x—x’). By the method of partial fractions, it can 
be shown that x/(1—x—x?) = €/V5)[1/(1—ax)-1/(1—fx)], 
where «a = (1 + /5)/2 and f = (1 - /5)/2. Us- 
ing the fact that 1/(1 — ax) = Ye?) aks‘, it follows that 
Ga) = (1/V5) - EZ — px. Hence, fe = (1/75) - 
(ak — p), 43. a)Let Gx) = YL) C,x" be the generat- 
ing function for {C,,}. Then G(x)? =) arose -o CeO) = 
ys Cr Cr! Se ea) = 
>, Cx”, which implies that xG(x)? — G(x) + 1 = 0. Ap- 


plying the quadratic formula shows that G(x) = ER 
We choose the minus sign in this formula because the choice 
of the plus sign leads to a division by zero. b) By Exer- 
cise 42, (1 — 4x)! = SD e graa Integrating term by 
term (which is valid by a theorem ‘from calculus) shows that 


x =1/2 _ co 1 (2n\ n _ 1 /2n\ n 
ha - 40) 1/ dt = D jin )x = =x) Taa ja 
Because f (1 — 4071?dt = Lvi = xG(x), equating 
coefficients shows that C, = < Cs c) Verify the basis 


step forn = 1, 2, 3, 4,5. Assume the inductive hypothe- 


sis that C, > 2! forl < j < n,wheren > 6. Then 


Cn = Lie R m= Èp mene C,- 20-22 2 = 


(n — 2)2""'/4 > 2"-!, 45. Applying the binomial theo- 
rem to the equality (1 + x»)"*" = (1 + x)"(1 + x)", shows 
that ey C(m + n, r)x"= Yo", Cm, rx- Fy C, r) x” = 
2o pa C(m,r—k)C(n, k)] x”. Comparing ls 
Dro gives the desired identity. 47. a) 2e” b)e™ c) e” 
d)xe*+e* 49.a)a, =(-1)" b)a, = 3-2” ¢) a, =3"-3-2" 
d) a, = (—2)" for n > 2, a, = —3,aọ =2 e)a, =(—2)" +n! 
f) a, = (—3)"+n!-2" forn > 2, ag = 1,a, = —2 g) a, = Oifnis 
oddanda, =n!/(n/2)!ifniseven 51. a) a„ = 6a,_,+8""! 
forn > 1,d) = 1 b) The general solution of the associ- 


ated linear homogeneous recurrence relation is a = a". 


A particular solution is an = 


3 - 8”. Hence, the general 
1 - 8”. Using the initial condition, 
, = (6" + 8")/2. ©) Let 
G(x) = Ye, qx*. Using the recurrence relation for {a,}, it 
can be shown that G(x) — 6xG(x) = (1 — 7x)/(1 — 8x). Hence, 
G(x) = (1 — 7x)/[U. — 6x). — 8x)]. Using partial fractions, 
it follows that Gœ) = (1/2)/(. — 6x) + (1/2)/(. — 8x). 
With the help of Table 1, it follows that a, = (6” + 8")/2. 
53. — 55. (14+x)(1+x)(1+x)3-- 57. The 
generating functions obtained in Exercises 54 and 55 are equal 
because (1+ 01 +A +8)- = SEES EY a 
1 1 1 


Ix le 1 59. a) Gx(D = Xio pX =k) 1 = 
YepPK=H=1 b) GD) = £ YP pK =H 41 = 


Eor = k-k- are = ae p(X =k)-k = EX) 


solution is a, = a6” + 


it follows thata = }. Hence, a, 
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GA) = E DPX = hx = DR pk = 
kK) -kk = 1) +x?) = De p(X =k) (Pr - k) = VX) + 
E(X} — E(X). Combining this with part (b) gives the desired 
results. 61. a) G(x) = p”/(. — qx)” b) V(x) = mq/p* 


Section 8.5 


1. a)30 b)29 c¢)24 d)18 3.1% 5.a) 300 b) 150 
ce) 175 d)100 7.492 9.974 11.610 13.55 15.248 
17.50,138 19.234 21. |A, UA, UA; UA, U A;| = 
|A | + 1A] + A3] + [A4] + IAs] — |A; 99] = |A; 9A3/— |A; N 
Al = 1A, NAs|— |42 N43] = |42 N44] = |42 N45] = |43 NAG] — 
|A; NA5|-— |44 N45] + |4; AA; NA;| + |A; NA, NA] + 1A, N 
A NA5|+ |4 NANA] +A; NA; NA5|+ |A; NA NA;|+]A N 
A3 NA, |+ |ANA; NA; |+|A NA} NA;5|+ |A; NA4NA;|-]A N 
A,NA3NA,|— |A; NA, NA; NA5| -|A NA, NA; NA;| -IA N 
A;NA4NAS|— |A, NA; NA, NA;5| + |4; NA, NA; NA4NA;| 
23. |A; UA, UA; UA} UA; UA] = |A |+ |A| +143] + [Ag] + 
IAs] + Ag] — |4; NAQ| — |A; NASI — JA, NAGI — |A; NAS] — 
|A; NAg| — [A42 NA3| = [A2 N44] — [A2 N45] = [A2 NAG] — 1439 
A4| — |A3 NAs5| — |A3 NA6| — [Ag NAs] — [A4 N46] — [As NAsl 
25. p(E, UE, U E3) = pE) + p(Ey) + p(E3) — pE N Ey) — 
pE, NE3) -pE NE3)+p(E, NE, NE3) 27. 4972/71,295 
29. P(E, UE, VE; VE, VEs) = p(E,)+p(E,) + p(E3) +p(Ey) + 
p(Es) — p(E, n E) — pE; n E3) — P(E, N Ey) — pE; N Es) — 
P(E, NE3)— pE NE4)— P(E, N Es) — p(E3 NE4) — p(E3N Es) — 
P(E, N Es) +p NE, NE) +pE NE, NE) + p(E, NEN 
Es)+p(E, NE3NEy)+p(E, NE4NEs) + P(E NE, NEs)+P(EsN 
E; Ey) + P(E) n E; Es) +p(E N Ey NEs) + p(Ey N Ey NEs) 
31. p (Ui E) = LicienPE) - LisicienP(E; N Ej) + 
Disi<jcken PCE; NE; N Ey) — + + Clr pit hak) 


Section 8.6 


1.75 3.6 5.46 7.9875 9.540 11.2100 13. 1854 
15. a) Do) /100! b) 100D 9/100! c) C(100,2)/100! 
d)O e)1/100! 17.2,170,680 19. By Exercise 18 we 


have D, — nD,_; = —[D,_; — (n — 1)D„-2]. Iterating, we 
have D,, — nD,_, = —[D,_;-(a—D)D,,_2] = —[-(D,_2 — (n - 
2)D ,,-3)] = D,-2 ~ (n ~ 2)D 1-3 Ss =. (-1)"D, = 2D;) = 
(—1)" because D, = 1 and D} = 0. 21. When n is odd 
23. on) = n— Ži Pi + D icigen Pp; SE Piba Pn = 


nJ (a - +) 25.4 27. There are n” functions from a 
set with m elements to a set with n elements, C(a, 1)(n — 1)” 
functions from a set with m elements to a set with n elements 
that miss exactly one element, C(n, 2)(n — 2)” functions from 
a set with m elements to a set with n elements that miss ex- 
actly two elements, and so on, with C(n, n — 1) - 1” functions 
from a set with m elements to a set with n elements that miss 
exactly n — 1 elements. Hence, by the principle of inclusion— 
exclusion, there are n” — C(n, 1)(n — 1)” + C(n, 2)(n — 2)” — 
+ + (-1)"!C(n, n — 1) - 1” onto functions. 


Supplementary Exercises 


1l. a)A, = 4A,_, b)A; = 40 oA, = 10- 4" 
3.a)M, = M,_; + 160,000 b)M, = 186,000 c)M, = 
160,000n + 26,000 d)7, = T,_, + 160,000n + 26,000 
e)T, = 80,000n? + 106,000n 5.a)a, = dy» + Ay-3 
b) a, 0, ay 1, a3 1 cap = 12 7.a)2 b)5 
c)8 d)16 9.a,=2" 11.a, =2+4n/3+n?/2+n/6 
13. a, =a,_,+4,_3; 15. a) Under the given conditions, one 


longest common subsequence ends at the last term in each 
sequence, so a,, = b, = c,. Furthermore, a longest com- 
mon subsequence of what is left of the a-sequence and the 
b-sequence after those last terms are deleted has to form the 
beginning of a longest common subsequence of the original 
sequences. b) If c, # a,,, then the longest common subse- 
quence’s appearance in the a-sequence must terminate before 
the end; therefore, the c-sequence must be a longest common 
subsequence of a), dy, ..., d,_; and b, bz, ..., b,,. The other 
half is similar. 
17. procedure howlong(a,, ...,d,,, bi, ..., b,: Sequences) 
for i := 1 tom 
Li, 0) := 0 
for j:=1ton 
L(O, j) := 0 
for i := 1 to m 
for j := 1 ton 
if a; = b; then LG, j) := Li — 1,j— 1) +1 
else L(i, j) := max(L(i, j — 1), L(i — 1, j)) 
return L(m, n) 
19. f(n) = (4n? — 1)/3 21.0(n*) 23. O(n) 25. Using 
just two comparisons, the algorithm is able to narrow the 
search for m down to the first half or the second half of the 
original sequence. Since the length of the sequence is cut in 
half each time, only about 2 log, n comparisons are needed 
in all. 27.a) 187 + 18 b)18 ©0 29. A(a,b,) 
an+lbn41— nbn = anlOn — Dn) + BO) (Qna, — 4) = 
a,,,Ab, + b,Aa, 31.a)Let Ga) = È% a,x". Then 


Gx) = Dna x! = Ye (a + Da, P, Therefore, 
G'(x) — Gx) = Pla + Dan -an = Daon! = e, 
as desired. That G(0) = dg = 1 is given. b) We have 
[e“G@)]’ = e*G'(x) — e*G(x) = eia) — Gx) = 


HX, pk 


e™. e* = 1. Hence, e*G(x) = x + c, where c is a con- 
stant. Consequently, G(x) = xe* + ce*. Because G(0) = 1, 


it follows thatc = 1. c) We have G(x) = YY") x1 /n!+ 


yo e/a! = Ya /@ — D! + DO x7 /ni. Therefore, 
a, = 1/(n — 1)! + 1/n! forall n > 1,anday = 1. 33.7 
35.110 37.0 39.a)19 b)65 c)122 d) 167 e) 168 
41. D,ı/(n— 1)! 43.11/32 


CHAPTER 9 


Section 9.1 


1. a) {(0, 0), CL 1, 2 2) (3, 3} DIG 3), @ 2), 
(3, 1), (4 0)} ©) {C, 9), (2, 0), 2 D, G, 0), G, 1), G, 2), 
(4, 0), (4, 1), (4 2), (4 3)} dd) {C1 0), (, 1), (G, 2), (1, 3), 
(2, 0), (2, 2), (3, 0), G, 3), (4, 0)} &) {0, D, G, 0), G, 1, 
d, 2), (1, 3), (2, D, (2, 3) (3, D, (3, 2), (4, D, 4 3)} 
f {d, 2), (2, 1), (2, 2)} 3. a) Transitive b) Reflexive, 
symmetric, transitive c€) Symmetric d) Antisymmetric 
e) Reflexive, symmetric, antisymmetric, transitive 
f)None of these properties 5.a) Reflexive, transi- 
tive b) Symmetric c) Symmetric d) Symmetric 
7.a) Symmetric b) Symmetric, transitive €) Symmetric 
d) Reflexive, symmetric, transitive e) Reflexive, transi- 
tive f) Reflexive, symmetric, transitive g) Antisymmetric 
h) Antisymmetric, transitive 9. Each of the three properties 
is vacuously satisfied. 11. (c), (d), (€ 13. a) Not irreflex- 
ive b) Not irreflexive c) Not irreflexive d) Not irreflex- 
ive 15. Yes, for instance {(1, 1)} on {1,2} 17. (a, b) € 
R if and only if a is taller than b 19.(a) 21. None 
23.VaVb [(a,b) E R > (b,a) é R] 25.2" 
27.a) {(a, b) | b divides a} b) {(a, b) | a does not di- 
videb} 29. The graph of f~! 31. a) {(a, b) | a is required 
to read or has read b} b) {(a, b) | a is required to read and 
has read b} œ) {(a, b) | either a is required to read b but has 
not read it or a has read b but is not required to} d) {(a, b) | a 
is required to read b but has not read it} e) {(a, b) | a has 
read b but is not required to} 33.S°R = {(a, b) | aisa 
parent of b and b has a sibling}, RoS = {(a, b) | ais an 
aunt or uncle of b} 35.a)R? b)R, © R, d)R, © Ø 
ÐR, gR, h)R, 37.a)R,; b)R, ©) R, d) R? e)R, 
f) R? g)R? h)R? 39. S? ={(a@ab €Z |a>b+1}, 
S = Sn $ = (ab) E€ Z |a < b-1},8% = Sy, 
= SS = Zz? 4l.b got his or her doctorate under 
someone who got his or her doctorate under a; there is a 
sequence of n + 1 people, starting with a and ending with 
b, such that each is the advisor of the next person in the se- 
quence 43.a) {(a, b) | a — b = 0, 3, 4, 6, 8, or 9 (mod 12)} 
b) {(a, b) | a = b (mod 12)} ©) {(4b) | a-b = 3,6, 
or 9 (mod 12)} d){(a,b) | a— b = 4or 8 (mod 12)} 
e) {(aa b) | a—b = 3,4,6,8, or 9 (mod 12)} 45.8 
47.a) 65,536 b)32,768 49.a) 27 D/2_— py 2”3”®-)/2 
c) 3n(n—1)/2 d) gnn-1) e) gn(n-1)/2 f) 20° _2.Qnn-1) 51. There 
may be no such b. 53. If R is symmetric and (a, b) € R, 
then (b, a) € R, so (a, b) € R`}. Hence, R C R7!. Similarly, 
R! C R. So R = R™!. Conversely, if R = R7! and (a, b) € R, 
then (a, b) € R!, so (b,a) € R. Thus, R is symmetric. 
55. R is reflexive if and only if (a, a) € R foralla € A if 
and only if (a, a) € R`! [because (a, a) € R if and only if 
(a, a) € R~'] if and only if R7! is reflexive. 57. Use mathe- 
matical induction. The result is trivial for n = 1. Assume R” is 
reflexive and transitive. By Theorem 1, R"+! C R. To see that 
R C R"+! = R"oR, let (a, b) € R. By the inductive hypothesis, 
R” = R and hence, is reflexive. Thus, (b, b) € R”. Therefore, 


Un 
D 
l 
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(a, b) € R"*!. 59, Use mathematical induction. The result 
is trivial for n = 1. Assume R” is reflexive. Then (a, a) E€ R” 
for all a € A and (a, a) € R. Thus, (a, a) E R"oR = R"*! for 
alla EA. 61. No, for instance, take R = {(1, 2), (2, 1)}. 


Section 9.2 


1. {(1, 2, 3), (1, 2, 4), (1, 3, 4), (2, 3,4)} 3. (Nadir, 122, 34, 
Detroit, 08:10), (Acme, 221, 22, Denver, 08:17), (Acme, 122, 
33, Anchorage, 08:22), (Acme, 323, 34, Honolulu 08:30), 
(Nadir, 199, 13, Detroit, 08:47), (Acme, 222, 22, Denver, 
09:10), (Nadir, 322, 34, Detroit, 09:44) 5. Airline and flight 
number, airline and departure time 7. a) Yes b) No c) No 
9. a) Social Security number b) There are no two people with 
the same name who happen to have the same street address. 
c) There are no two people with the same name living to- 
gether. 11. (Nadir, 122, 34, Detroit, 08:10), (Nadir, 199, 13, 
Detroit, 08:47), (Nadir, 322, 34, Detroit, 09:44) 13. (Nadir, 
122, 34, Detroit, 08:10), (Nadir, 199, 13, Detroit, 08:47), 
(Nadir, 322, 34, Detroit, 09:44), (Acme, 221, 22, Denver, 
08:17), (Acme, 222, 22, Denver, 09:10) 15. P3 56 


17. Airline 


Nadir 
Acme 
Acme 
Acme 
19. 

Supplier 

23 

23 

23 

31 

31 

32 

32 

33 


21. Both sides of this equation pick out the subset of R con- 
sisting of those n-tuples satisfying both conditions C} and C3. 
23. Both sides of this equation pick out the set of n-tuples 
that are in R, are in S, and satisfy condition C. 25. Both 
sides of this equation pick out the m-tuples consisting of 
ith, ith, ..., i„th components of n-tuples in either R or S. 
27. Let R = {(a, b)} and S = { (a, c)},n=2, m= 1,andi, = 1; 
P (R — $) = {(a)}, but P,(R) — P\(S) = Ø. 29. a) J, fol- 
lowed by P, 3 b) (23, 1), (23, 3), (31, 3), (32,4) 31. There is 
no primary key. 33. a) count 5, support 5/6 b) {diapers}, 
{milk}, and {diapers, milk} 35. support 1/3, confidence 
2/3 37.1 39.a)PU) = o()/|T|, PY) = o()/|TI, 
P(I and J) = o([UJ)/|T|, so PU and J) = P(I)-P(J) if and only 
if ol UJ)/|T| = (6Z)/|T|) - (@V)/|T|), which means that 
the numerator and denominator of the lift are equal. b) 4/5 
c) 16/15 41. For each of the n items, there are 3 choices: 
the item can be in / or in J or in neither. By the product rule, 
there are 3” ways to make these decisions. 


Destination 


Detroit 
Denver 
Anchorage 
Honolulu 
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Section 9.3 
Laji 1 1] Dlo 1 0 
0 0 0 1 1 0 
0 0 0 00 1 
Ol1 1 1] ®fo o 1 
0 i I 0 0 0 
00 1 10 0 


3. a) (1, 1), CG, 3), @ 2), GB, 1, 3, 3) b) Cd, 2), (2, 2), 
(3,2) ©), 1), (1 2), (1, 3), 2 1), 2 3), G, 1), 6, 2), G, 3) 
5. The relation is irreflexive if and only if the main diago- 
nal of the matrix contains only Os. 7. a) Reflexive, sym- 
metric, transitive b) Antisymmetric, transitive €) Symmetric 
9. a) 4950 b) 9900 c) 99 d) 100 e)1 11. Change each 0 
to a | and each 1 toa 0. 


13. ajo 1 1] bļ|ı O Of oji 
1 1 0 0 0 1 1 
0 1 


1 0 1 0 1 
15. a)]0 0 b)i 1 0| ©j0 
t 1 0 1 1 
1 1 1 


Cor 
— 


© 1 1 
17. n? —k 


19. a)! 4 
x 

c) 1 4 d) ! 4 
hq X 


21. For simplicity we have indicated pairs of edges between 


the same two vertices in opposite directions by using a double 
arrowhead, rather than drawing two separate lines. 


A 


. {(a, b), (a, c), (b, c), (c, b)} 25. (a, c), (b, a), (c, d), 
a b) 27. {(a, a), (a, b), (a, c), (b, a), (b, b), (b, c), (c, a), 
(c, b), (d, d)} 29. The relation is asymmetric if and only 
if the directed graph has no loops and no closed paths of 
length 2. 31. Exercise 23: irreflexive. Exercise 24: reflex- 
ive, antisymmetric, transitive. Exercise 25: irreflexive, anti- 
symmetric. 33. Reverse the direction on every edge in the 
digraph for R. 35. Proof by mathematical induction. Basis 
step: Trivial for n = 1. Inductive step: Assume true for k. Be- 
cause R*+! = RÝ o R, its matrix is M M OM. By the inductive 
hypothesis this is Mp © M! = e, 


Section 9.4 


1. a) {(0, 0), (0, 1), (1, 1), (1, 2), (2, 0), (2, 2), (3, 0), (3, 3)} 
b) {(0, 1), (0, 2), (0, 3), (1, 0), CL 1), CL 2), (2, 0), (2, 1), (2, 2), 
(3,0)} 3. {(a, b) | a divides b or b divides a} 


11. a) 


13. The symmetric closure of R is R U R™!. Maug = 
Mr V Mer = Mr V Mẹ- 15. Only when R is irreflex- 
ive, in which case it is its own closure. 17. a, a, a, a; a, b, 
e, a; a, d, e, a; b, c, c, b; b, e a, beb eee © beig 
c, c, c; d, e, a, d, d, e, e, d; e, a, b, e; e, a, d, e; e, d, e, e; e, 
e d, e;e,e,e,e 19.a){(1, 1), (1, 5), (2, 3), (3, 1), (3, 2), 
(3, 3), 3, 4), (4, 1), (4, 5), (5, 3), 6, 9} b) {0, 1), CL. 2), 
(13), (L-4, 2 1), 2 5), 6.1), 6.3), 8,4), 6,5), (4 1), 
(4, 2), (4 3), 4 4), 6 D, 6, 3), 65,5) © {0 1), CL. 3), 
G, 4), A, 5), 2 1), @, 2), 2 3), 2, 4), G, 1), 683, 2), G, 3), 
(3, 4), (3,5), 4-1), 4 3), 44, 4 5). 6, D, G, 2), G, 3), 
(5, 4), (5, 5)} d) {CL 1), (, 2), C, 3), (, 4), (, 5), (2 1), 
(2, 3), (2, 4), (2, 5), 3, 1), G, 2), 6, 3), G, 4), G, 5), (4 1), 
(4, 2), (4, 3), (4 4), (4 5), 6, D, 5, 2), , 3), 5, 4), 5, 5)} 
e) {(, 1), C, 2), (1, 3), CL 4), (1, 5), (2, 1), (2, 2), (2, 3), (2, 4), 
(2, 5), (3, D, G, 2), B, 3), (3, 4, B, 5), (4 1), (4 2), 4 3), 
(4, 4), (4 5), G, 1), G, 2), G, 3), 6. 4, 6,5} {CL 1, 
(1, 2), (1, 3), GL 4), CL 5), (2 1), 2 2), 2 3), (2 4), (2 5), 
(3, 1), G, 2), B, 3), G, 4, G, 5), (4, 1), 4 2), (4 3), (4 4), 
(4, 5), (5, 1), (5, 2), (5, 3), (5, 4), (5,5)} 24. a) If there is a 
student c who shares a class with a and a class with b b) If 
there are two students c and d such that a and c share a class, 
c and d share a class, and d and b share a class c) If there is a 
sequence Sp, ..., S, of students with n > 1 such that sy = a, 


Ss, = b, and for each i = 1,2,...,n, s; and s,;_, share a 


class 23. The result follows from (R*)~! = (US, Rr") = 
UZ R! = J R" = R*. 


25. aji 1 1 1] Dio o 0 o 
tf 4 1 4 1 0 1 1 
Lii i01 l 
Íi i í a 1 0 1 1 

Ofo 1 1 al®jh 1 1 1 
0011 l l1 1 1 
000 1 idil 
0000 l 1 1 1 


27. Answers same as for Exercise 25. 29. a) {(1, 1), (1, 2), 
(1, 4), (2, 2), (3, 3), (4, 1), (4, 2), (4, 9} b) (CL D, (L, 2), (1, 4), 
(2, 1), @ 2), 2, 4, G, 3), (4, D, (4, 2),4 9} O fC, 1), 
C, 2), (L 4), (2 1), 2 2), 24, (3, 3), (4 1), (4 2), (4, 4)} 
31. Algorithm 1: O(n?8); Algorithm 2: O(n?) 33. Initialize 
with A := Me V I, and loop only fori := 2 ton — 1. 
35. a) Because R is reflexive, every relation containing it must 
also be reflexive. b) Both {(0, 0), (0, 1), (0, 2), (d, 1), (2, 2)} 
and {(0, 0), (0, 1), (1, 0), (1, 1), (2, 2)} contain R and have an 
odd number of elements, but neither is a subset of the other. 


Section 9.5 


1. a) Equivalence relation b) Not reflexive, not transitive 
c) Equivalence relation d) Not transitive e) Not symmetric, 
not transitive 3. a) Equivalence relation b) Not transitive 
c) Not reflexive, not symmetric, not transitive d) Equivalence 
relation e) Not reflexive, nottransitive 5. Many answers are 
possible. (1) Two buildings are equivalent if they were opened 
during the same year; an equivalence class consists of the set 
of buildings opened in a given year (as long as there was at 
least one building opened that year). (2) Two buildings are 
equivalent if they have the same number of stories; the equiv- 
alence classes are the set of 1-story buildings, the set of 2- 
story buildings, and so on (one class for each n for which there 
is at least one n-story building). (3) Every building in which 
you have a class is equivalent to every building in which you 
have aclass (including itself), and every building in which you 
don’t have a class is equivalent to every building in which you 
don’t have a class (including itself); there are two equivalence 
classes—the set of buildings in which you have a class and 
the set of buildings in which you don’t have a class (assuming 
these are nonempty). 7. The statement “p is equivalent to q” 
means that p and q have the same entries in their truth tables. 
R is reflexive, because p has the same truth table as p. R is 
symmetric, for if p and q have the same truth table, then q and 
p have the same truth table. If p and q have the same entries 
in their truth tables and g and r have the same entries in their 
truth tables, then p and r also do, so R is transitive. The equiv- 
alence class of T is the set of all tautologies; the equivalence 
class of F is the set of all contradictions. 9. a) (x, x) E€ R 
because f(x) = f(x). Hence, R is reflexive. (x, y) € Rif and 
only if f(x) = f(y), which holds if and only if f(y) = f(x) if 
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and only if (y, x) € R. Hence, R is symmetric. If (x, y) E€ R 
and (y, z) E R, then f(x) = f(y) and f(y) = f(z). Hence, 
f(x) = f(z). Thus, (x, z) E€ R. It follows that R is transitive. 
b) The sets f~!(b) for b in the range of f 11. Let x be a bit 
string of length 3 or more. Because x agrees with itself in the 
first three bits, (x, x) € R. Hence, R is reflexive. Suppose that 
(x, y) E R. Then x and y agree in the first three bits. Hence, 
y and x agree in the first three bits. Thus, (y, x) E€ R. If (x, y) 
and (y, z) are in R, then x and y agree in the first three bits, as 
do y and z. Hence, x and z agree in the first three bits. Hence, 
(x, z) E R. It follows that R is transitive. 13. This follows 
from Exercise 9, where f is the function that takes a bit string 
of length 3 or more to the ordered pair with its first bit as 
the first component and the third bit as its second component. 
15. For reflexivity, ((a, b), (a, b)) E R because a + b = b + a. 
For symmetry, if ((a, b), (c, d)) € R, thena + d = b +c, 
soc +b = d + a, so ((c, d), (a, b)) € R. For transitivity, if 
((a, b), (c, d)) € R and ((c, d), (e, f)) E R, then a+d = b+c and 
c+e=d+f,soa+d+c+e=b+c+d+f,soa+e=b+f, 
so ((a, b), (e, f)) € R. An easier solution is to note that by 
algebra, the given condition is the same as the condition that 
f(a b)) = f((c, d)), where f((x, y)) = x — y; therefore, by 
Exercise 9 this is an equivalence relation. 17. a) This fol- 
lows from Exercise 9, where the function f from the set of 
differentiable functions (from R to R) to the set of functions 
(from R to R) is the differentiation operator. b) The set of 
all functions of the form g(x) = x? + C for some constant C 
19. This follows from Exercise 9, where the function f from 
the set of all URLs to the set of all Web pages is the func- 
tion that assigns to each URL the Web page for that URL. 
21.No 23.No 25. Ris reflexive because a bit string s has 
the same number of 1s as itself. R is symmetric because s and 
t having the same number of 1s implies that ¢ and s do. R is 
transitive because s and ¢ having the same number of 1s, and 
t and u having the same number of 1s implies that s and u 
have the same number of ls. 27. a) The sets of people of 
the same age b) The sets of people with the same two parents 
29. The set of all bit strings with exactly two ls 31. a) The 
set of all bit strings of length 3 b) The set of all bit strings 
of length 4 that end with a 1 c) The set of all bit strings of 
length 5 that end 11 d) The set of all bit strings of length 8 
that end 10101 33. Each of the 15 bit strings of length less 
than four is in an equivalence class by itself: [A]p, = {A}, 
Ne, = {0}, He, = {1}, [00]p, = {00}, [01]g, = {01}, 
.-. [I11]p, = {111}. The remaining 16 equivalence classes 
are determined by the bit strings of length 4: [0000],, = 
{0000, 00000, 00001, 000000, 000001, 000010, 000011, 
0000000, ...}, [0001]z, = {0001, 00010, 00011, 000100, 
000101, 000110, 000111, 0001000, ...}, ..., [I111]p, = 
{1111, 11110, 11111, 111100, 111101, 111110, 111111, 
1111000, ...} 35.a)[2]; = {i | i=2 (mod 5)}} = 
{..., —8, -3, 2, 7, 12,...} b Bl = {i | i = 3 
(mod 5)} = {..., -7, —2, 3, 8, 13, ...} ©) [6]; = {i |i = 6 
(mod 5)} = {..., —9, —4, 1,6, 11, ...} d) [-3]; = {i | i = —3 
(mod 5)} = {..., —8, —3, 2, 7, 12, ...} 37. {6n+k|n € Z} 
fork € {0, 1, 2,3,4,5} 39.a)[(1,2)] = {(a, b) | a — b = 
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-1} = {(1, 2), (3, 4), (4, 5), (5, 6), ...} b) Each equiva- 
lence class can be interpreted as an integer (negative, posi- 
tive, or zero); specifically, [(a, b)] can be interpreted as a — b. 
41.a)No b) Yes c) Yes d)No 43. (a), (c), (e) 45. (b), 
(d), (e) 47. a) {(0, 0), G, 1), (1, 2), 2 D, 2, 2), G, 3), G, 4), 
(3, 5), (4, 3), (4, 4), (4 5), (5, 3), G, 4, S, 5)} b) {(0, 0), 
0, 1, C, 0), G, D, @ 2), 2 3) G, 2) GB, 3) (4, 4, 
(4, 5), GS, 4), (5, 5)} œ {0, 0), ©, 1), ©, 2), CL 0), CL 1), 
a, 2), (2, 0), 2 D, @ 2) GB, 3) G, 4} G, 5) 4 3), 
(4, 4), (4, 5), G, 3), S, 4), S, 5)} d) {0, 0), CL D, (2 2), 
(3, 3), (4,4), (5, 5)} 49. [0]6 € [0]s, [1]6 E Hs, [2]6 E [2], 
[3], © [0], [4], E [1], [5] E [2]; 51. Let A be a set 
in the first partition. Pick a particular element x of A. The set 
of all bit strings of length 16 that agree with x on the last four 
bits is one of the sets in the second partition, and clearly every 
string in A is in that set. 53. We claim that each equivalence 
class [x] Ry is a subset of the equivalence class [x] Re: To show 
this, choose an arbitrary element y € [x]p,. Then y is equiv- 
alent to x under R;,, so either y = x or y and x are each at 
least 31 characters long and agree on their first 31 characters. 
Because strings that are at least 31 characters long and agree 
on their first 31 characters perforce are at least 8 characters 
long and agree on their first 8 characters, we know that either 
y =x ory and x are each at least 8 characters long and agree 
on their first 8 characters. This means that y is equivalent to x 
under Rg, soy E [x], 55. {(a, a), (a, b), (a, c), (b, a), (b, b), 
(b, c), (c, a), (c, b), (c, c), (d, d), (d, e), (e, d), (e,e)} 57.a) Z 
b) {n++ |n€Z} 59. a) Risreflexive because any coloring 
can be obtained from itself via a 360-degree rotation. To see 
that R is symmetric and transitive, use the fact that each rota- 
tion is the composition of two reflections and conversely the 
composition of two reflections is a rotation. Hence, (C4, C ) 
belongs to R if and only if C, can be obtained from C, by a 
composition of reflections. So if (C,, C,) belongs to R, so does 
(Cz, C1) because the inverse of the composition of reflections 
is also a composition of reflections (in the opposite order). 
Hence, R is symmetric. To see that R is transitive, suppose 
(C,, C,) and (C,, C3) belong to R. Taking the composition of 
the reflections in each case yields a composition of reflections, 
showing that (C,, C3) belongs to R. b) We express colorings 
with sequences of length four, with r and b denoting red and 
blue, respectively. We list letters denoting the colors of the up- 
per left square, upper right square, lower left square, and lower 
right square, in that order. The equivalence classes are: {rrrr}, 
{bbbb}, {rrrb, rrbr, rbrr, brrr}, {bbbr, bbrb, brbb, rbbb}, 
{rbbr, brrb\, {rrbb, brbr, bbrr, rbrb}. 61.5 63. Yes 
65.R 67. First form the reflexive closure of R, then form the 
symmetric closure of the reflexive closure, and finally form 
the transitive closure of the symmetric closure of the reflex- 
ive closure. 69. p(0) = 1, p(1) = 1, p(2) = 2, p(3) = 5, 
P(A) = 15, p(5) = 52, p(6) = 203, p(7) = 877, p(8) = 4140, 
p(9) = 21147, p(10) = 115975 


Section 9.6 


1. a) Is a partial ordering b) Not antisymmetric, not transitive 
c) Is a partial ordering d) Is a partial ordering e) Not antisym- 
metric, not transitive 3.a)No b)No c) Yes 5.a) Yes 
b) No c) Yes d)No 7.a)No b) Yes c)No_ 9.No 
11. Yes 13. a) {(0, 0), (1, 0), (1, 1), (2, 0), (2, 1), (2, 2)} 
b) (Z, <) ¢) (P(Z), ©) d) (Z+, “is a multiple of’) 15. a) {0} 
and {1}, for instance b) 4 and 6, for instance 17. a) (1, 1, 2) < 
(1,2,1) b)(0, 1,2,3) < (0,1,3,2) œc)(0, 1,1,1,0) < 
(1,0,1,0,1) 19.0 < 0001 < 001 < 01 < 010 < 0101 < 
011 < 11 


21. be 
110 
10% 
50 
20 
00 
23. a) 8 b)2 3 5 7 ll B 
43 “a~ 
2 5 7 l 
1 
c) 480 d) 964 
24% > @32 
Y @ 16 
6 $8 
2 3 $4 
i @2 
el 


25. (a, b), (a, c), (a, d), (b, c), (b, d), (a, a), (b, b), (c, c), 
(d, d) 27. (a, a), (a, 8), (a, d), (a, e), (a, f), (b, b), (b, 8), 
(b, d), (b, e), (b, f), (c, c), (c, 8), (c d), (c, e), (c f), (8 d), 
(8, e), (8, f), 8 8), (d, d), (e, e), Ff) 29.G, {a}), 
Ø, {b}), Ø, {c}), La}, {a, b}), Ca}, {a, ch), Ab}, (a, b}), 
({b}, {b, c}), Ae}, {a, c}), Ce}, {b, ch), Ea b}, {a b, c}), 
({a, c}, {a b, c})C{b, c}, {a,b,c}) 31. Let (S, =) be a finite 
poset. We will show that this poset is the reflexive transi- 
tive closure of its covering relation. Suppose that (a, b) is in 
the reflexive transitive closure of the covering relation. Then 
a = bora < b,soa = b, or else there is a sequence 
dy, Gy, ..., a, Such thata < a; < ad) < © < a, < b, 
in which case again a = b by the transitivity of =. Con- 
versely, suppose that a < b. If a = b then (a, b) is in the 


reflexive transitive closure of the covering relation. If a < b 
and there is no z such thata < z < b, then (a, b) is in 
the covering relation and therefore in its reflexive transitive 
closure. Otherwise, leta < a, < a, < > < a, < b 
be a longest possible sequence of this form (which exists be- 
cause the poset is finite). Then no intermediate elements can 
be inserted, so each pair (a, a4), (ay, a2), ..., (a, b) is in the 
covering relation, so again (a, b) is in its reflexive transitive 
closure. 33.a)24,45 b)3,5 c)No d)No e) 15, 45 
15 g)15,5,3 h)15 35. a) {1, 2}, (1, 3, 4}, {2 3, 4} 
b) {1}, {2}, {4} ©) No d)No e) {2,4}, {2,3,4} Ð {2,4} 
g) {3,4}, {4} bh) {3,4} 37. Because (a, b) x (a, b), x is 
reflexive. If (a,, a2) x (b,, b2) and (a), dy) # (by, bp), either 
a, < b,, ora, = b; anda, < by. In either case, (b4, bz) 
is not less than or equal to (a,, a,). Hence, = is antisym- 
metric. Suppose that (a;, a2) < (by, b2) < (cy, c2). Then if 
a, < b; orb; < c}, we have a, < c}, 80 (ay, a) < (Cj, C2), 
but if a) = b; = c4, then a < by < c, which implies 
that (a), a) < (c1, Cy). Hence, = is transitive. 39. Because 
(s, £) x (s, £), < is reflexive. If (s, £) < (u, v) and (u, v) = (s, £), 
then s x u x sandt x v x t; hence, s = u and t = v. Hence, 
= is antisymmetric. Suppose that (s, t) <(u, v) x(w, x). 
Then s x u, t x v, u x w, and v x x. It follows that s x w 
and t =< x. Hence, (s, t) <(w, x). Hence, x is transitive. 
41. a) Suppose that x is maximal and that y is the largest el- 
ement. Then x = y. Because x is not less than y, it follows 
that x = y. By Exercise 40(a) y is unique. Hence, x is unique. 
b) Suppose that x is minimal and that y is the smallest ele- 
ment. Then x > y. Because x is not greater than y, it follows 
that x = y. By Exercise 40(b) y is unique. Hence, x is unique. 
43. a) Yes b) No c) Yes’ 45. Use mathematical induction. 
Let P(n) be “Every subset with n elements from a lattice has 
a least upper bound and a greatest lower bound.” Basis step: 
P(1) is true because the least upper bound and greatest lower 
bound of {x} are both x. Inductive step: Assume that P(k) 
is true. Let S be a set with k + 1 elements. Let x € S and 
S’ = S — {x}. Because S’ has k elements, by the inductive 
hypothesis, it has a least upper bound y and a greatest lower 
bound a. Now because we are in a lattice, there are elements 
z = lub(, y) and b = glb(x, a). We are done if we can show 
that z is the least upper bound of S and b is the greatest lower 
bound of S. To show that z is the least upper bound of S, first 
note that if w € S, then w = x or w E€ S’. If w = x then w 3 z 
because z is the least upper bound of x and y. If w € S’, then 
w x z because w = y, which is true because y is the least up- 
per bound of S’, and y = z, which is true because z = lub(x, y). 
To see that z is the least upper bound of S, suppose that u is 
an upper bound of S. Note that such an element u must be an 
upper bound of x and y, but because z = lub(x, y), it follows 
that z < u. We omit the similar argument that b is the great- 
est lower bound of S$. 47. a) No b) Yes c¢) (Proprietary, 
{Cheetah, Puma}), (Restricted, {Cheetah, Puma}), (Reg- 
istered, {Cheetah, Puma}), (Proprietary, {Cheetah, Puma, 
Impala}), (Restricted, {Cheetah, Puma, Impala}), (Reg- 
istered, {Cheetah, Puma, Impala}) d) (Nonproprietary, 
{Impala, Puma}), (Proprietary, {Impala, Puma}), 
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(Restricted, {Impala, Puma}), (Nonproprietary, {Impala}), 
(Proprietary, {Impala}), (Restricted, {Impala}), (Nonpro- 
prietary, {Puma}), (Proprietary, {Puma}), (Restricted, 
{Puma}), (Nonproprietary, Ø), (Proprietary, Ø), (Restricted, 
Ø) 49. Let II be the set of all partitions of a set S with 
P, x P, if P, isa refinement of P,, that is, if every set in P} 
is a subset of a set in P}. First, we show that (II, = ) is a poset. 
Because P = P for every partition P, = is reflexive. Now 
suppose that P, =< P, and P, = P,. Let T € P,. Because 
P, = Py, there is a set T’ € P, such that T C T’. Because 
P, = P; there is a set T” € P} such that T’ C T”. It follows 
that T C T”. But because P} is a partition, T = T”, which 
implies that T = T’ because T C T’ C T”. Thus, T € P}. 
By reversing the roles of P} and P, it follows that every set 
in P, is also in P}. Hence, P} = P, and = is antisymmet- 
ric. Next, suppose that P; = P, and P, x P}. Let T € P}. 
Then there is a set T’ € P, such that T C T’. Because 
P, = P; there is a set T” € P, such that T’ C T”. This 
means that T C T”. Hence, P} =< P}. It follows that = is 
transitive. The greatest lower bound of the partitions P; and 
P, is the partition P whose subsets are the nonempty sets of 
the form T, N T, where T) € P} and T, € P,. We omit the 
justification of this statement here. The least upper bound of 
the partitions P, and P, is the partition that corresponds to 
the equivalence relation in which x € S is related toy € S 
if there is a sequence x = Xp, Xj, Xz ...,%X, = y for some 
nonnegative integer n such that for each i from | to n, x; 
and x; are in the same element of P} or of P). We omit the de- 
tails that this is an equivalence relation and the details of the 
proof that this is the least upper bound of the two partitions. 
51. By Exercise 45 there is a least upper bound and a greatest 
lower bound for the entire finite lattice. By definition these 
elements are the greatest and least elements, respectively. 
53. The least element of a subset of Z* x Z* is that pair that 
has the smallest possible first coordinate, and, if there is more 
than one such pair, that pair among those that has the small- 
est second coordinate. 55. If x is an integer in a decreasing 
sequence of elements of this poset, then at most |x| elements 
can follow x in the sequence, namely, integers whose absolute 
values are |x| — 1, |x] — 2, ..., 1, 0. Therefore, there can be no 
infinite decreasing sequence. This is not a totally ordered set, 
because 5 and —5, for example, are incomparable. 57. To 
find which of two rational numbers is larger, write them with 
a positive common denominator and compare numerators. To 
show that this set is dense, suppose that x < y are two rational 
numbers. Then their average, i.e., (x+y)/2, is a rational num- 
ber between them. 59. Let (S, =) be a partially ordered set. 
It is enough to show that every nonempty subset of S contains 
a least element if and only if there is no infinite decreasing 
sequence of elements a}, dy, 43, ... in S (i.e., where aip; < aj 
for all 7). An infinite decreasing sequence of elements clearly 
has no least element. Conversely, let A be any nonempty sub- 
set of S that has no least element. Because A is nonempty, 
choose a, € A. Because a, is not the least element of A, 
choose a, € A with a, < a. Because a, is not the least 
element of A, choose a; E€ A with a; < dy. Continue in 
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this manner, producing an infinite decreasing sequence in S. 
61. a<,b <,c <,d <,e <,f <,¢<,h<,i<,j<,k<,l<,m 
63. 1<5 «2 «4 <« 12 ~ 20,1 «~2 ~5 <4 <« 12 < 20, 
1~«~ 2 <4 <5 <« 12 ~ 20,1 ~« 2 «~ 4 ~ 12 ~ 5 <« 20, 
1 <5 «2 ~« 4 ~ 20 K 12,1 x 2 <5 «K 4 « 20 ~« 12, 
1<2<4<5<20<12 65.A<C<E<B<D<F<G, 
A<E<C<B<D<F<«<G,C<A<E<B<D<F<G, 
CK<E~A<B<D<K«<F<«G,E<A<C<B<D<F<G, 
E<CK<A<B<D<F«G,A<C<BK<E<D<F<G, 
CKA<BKE<D<K«<F«<G,A<CK<BK<D<E<F<G, 
CK<A<BKXD<K<EKF«<G,A<CK<E<B<F<D<G, 
A<E<CH<BK«FK<D<G,C<A<E<B<F<D<G, 
CK<EKXA<BK<F<«<D<G,E~A<C<B<F<D<G, 
E<C~A<B<F<«<D<G,A<C<BK«<E<F<D<G, 
C<A-<B<E<«< F< D<G_ 67. Determine user 
needs < Write functional requirements < Set up test sites < 
Develop system requirements < Write documentation < De- 
velop module A < Develop module B < Develop module C < 
Integrate modules < a test < p test < Completion 


Supplementary Exercises 


1. a) Irreflexive (we do not include the empty string), symmet- 
ric b) Irreflexive, symmetric c) Irreflexive, antisymmetric, 
transitive 3. ((a, b), (a, b)) E R because a+b = a+b. Hence, 
R is reflexive. If ((a, b), (c, d) E R then a + d = b + c, so that 
c +b = d + a. It follows that ((c, d), (a, b)) E R. Hence, R is 
symmetric. Suppose that ((a, b), (c, d)) and ((c, d), (e, f)) be- 
long to R. Then a+d = b+c and c+f = d+e. Adding these two 
equations and subtracting c + d from both sides gives a +f = 
b+e. Hence, ((a, b), (e, f)) belongs to R. Hence, R is transitive. 
5. Suppose that (a, b) € R. Because (b, b) € R it follows that 
(a,b) € R?. 7. Yes, yes 9. Yes, yes 11. Two records 
with identical keys in the projection would have identical keys 
in the original. 13. (A UR)! = AUR! = AUR! 
15. a)R = {(a, b), (a, c)}. The transitive closure of 
the symmetric closure of R is {(a, a), (a, b), (a, c), (b, a), 
(b, b), (b, c), (c, a), (c, b), (c, c)} and is different from the 
symmetric closure of the transitive closure of R, which is 
{(a, b), (a, c), (b, a), (c, a)}. b) Suppose that (a, b) is in the 
symmetric closure of the transitive closure of R. We must 
show that (a, b) is in the transitive closure of the symmetric 
closure of R. We know that at least one of (a, b) and (b, a) 
is in the transitive closure of R. Hence, there is either a path 
from a to b in R or a path from b to a in R (or both). In the 
former case, there is a path from a to b in the symmetric clo- 
sure of R. In the latter case, we can form a path from a to b 
in the symmetric closure of R by reversing the directions of 
all the edges in a path from b to a, going backward. Hence, 
(a, b) is in the transitive closure of the symmetric closure of 
R. 17. The closure of S with respect to property P is a rela- 
tion with property P that contains R because R C S. Hence, 
the closure of S with respect to property P contains the clo- 
sure of R with respect to property P. ao Use the basic idea 
of Warshall’s algorithm, except let wit l equal the length of 
the longest path from v; to v, using interior vertices with sub- 


scripts not exceeding k, and equal to —1 if there is no such 
path. To find wil from the entries of W,_,, determine for each 


pair (i, j) whether there are paths from v; to v, and from v; to 
{k-1] o 
ik 


is —1, then such a pair of paths does not exist, so set 


vj using no vertices labeled greater than k. If either w; 
w lji; 
Wii 
wl 


w = w=, If such a pair of paths exists, then there are 


two possibilities. If wit] > 0, there are paths of arbitrary 


long length from v; to v,, ia 


so set wj = ©. If wt 
set wie 1 = marga 4 we de we My, (Initially take 
W) = Mg.) 21.25 23. Because A; a B; is a subset of 
A; and of B,, the collection of subsets is a refinement of each 
of the given partitions. We must show that it is a partition. By 
construction, each of these sets is nonempty. To see that their 
union is S, suppose that s € S. Because P} and P, are parti- 
tions of S, there are sets A; and B; such that s € A; and s E€ B}. 
Therefore, s € A; N B;. Hence, the union of these sets is S. To 
see that they are aie disjoint, note that unless i = i’ and 
j = J’, (A; 9B) A (Ay N By) = (A; N Ap) N (B; N By) = Ø. 
25. The subset relation is a ‘partial ordering on any collection 
of sets, because it is reflexive, antisymmetric, and transitive. 
Here the collection of sets is R(S). 27. Find recipe < Buy 
seafood < Buy groceries < Wash shellfish < Cut ginger and 
garlic < Clean fish < Steam rice < Cut fish < Wash vegetables 
< Chop water chestnuts < Make garnishes < Cook in wok < 
Arrange on platter < Serve 29. a) The only antichain with 
more than one element is {c,d}. b) The only antichains with 
more than one element are {b, c}, {c, e}, and {d, e}. c) The 
only antichains with more than one element are {a, b}, {a, c}, 
{b, c}, {a b, c}, {d,e}, {d, f}, {ef}, and {d,e,f}. 31. Let 
(S, x) be a finite poset, and let A be a maximal chain. Be- 
cause (A, x) is also a poset it must have a minimal element 
m. Suppose that m is not minimal in S. Then there would be 
an element a of S with a < m. However, this would make the 
set A U {a} a larger chain than A. To show this, we must show 
that a is comparable with every element of A. Because m is 
comparable with every element of A and m is minimal, it fol- 
lows that m < x when x is in A and x # m. Because a < m and 
m < x, the transitive law shows that a < x for every element of 
A. 33. Let aRb denote that a is a descendant of b. By Exer- 
cise 32, if no set of n+ 1 people none of whom is a descendant 
of any other (an antichain) exists, then k < n, so the set can 
be partitioned into k < n chains. By the pigeonhole princi- 
ple, at least one of these chains contains at least m+ 1 people. 
35. We prove by contradiction that if S has no infinite decreas- 
ing sequence and Vx ({Vyby <x > P) > P(x)), then P(x) 
is true for all x € S. If it does not hold that P(x) is true for 
all x € S, let x, be an element of S such that P(x,) is not 
true. Then by the conditional statement already given, it must 
be the case that Vy[y < x, — P(y)] is not true. This means 
that there is some x, with x, < x, such that P(x,) is not true. 
Again invoking the conditional statement, we get an x, < x, 
such that P(x3) is not true, and so on forever. This contradicts 
the well-foundedness of our poset. Therefore, P(x) is true for 
allx € S. 37. Suppose that R is a quasi-ordering. Because 
R is reflexive, if a € A, then (a, a) € R. This implies that 


(a, a) E R~. Hence, a E R N R™. It follows that RA R7! is 
reflexive. R N R7! is symmetric for any relation R because, 
for any relation R, if (a,b) € R then (b, a) € R`! and 
vice versa. To show that R n R7! is transitive, suppose that 
(a,b) € RNR and (b, c) € RNR“. Because (a, b) € R and 
(b,c) E R, (a,c) E R, because R is transitive. Similarly, be- 
cause (a, b) € R7! and (b, c) E€ R™!, (b, a) € Rand (c, b) E R, 
so (c, a) € R and (a,c) € R~. Hence, (a, c) E€ R N R7}. It fol- 
lows that RAR”! is an equivalence relation. 39. a) Because 
glb(x, y) = glb(, x) and lub(x, y) = lub(y, x), it follows that 
xAy =yAxandxVy =yVx. b) Using the definition, (xA y) Az 
is a lower bound of x, y, and z that is greater than every other 
lower bound. Because x, y, and z play interchangeable roles, 
XA AZ) is the same element. Similarly, (x Vy) Vz is an upper 
bound of x, y, and z that is less than every other upper bound. 
Because x, y, and z play interchangeable roles, xV (yV z) is the 
same element. c) To show that x A (x V y) = x it is sufficient 
to show that x is the greatest lower bound of x, and x V y. Note 
that x is a lower bound of x, and because x V y is by definition 
greater than x, x is a lower bound for it as well. Therefore, x is 
a lower bound. But any lower bound of x has to be less than 
x, So x is the greatest lower bound. The second statement is 
the dual of the first; we omit its proof. d) x is a lower, and an 
upper, bound for itself and itself, and the greatest, and least, 
such bound. 41. a) Because 1 is the only element greater 
than or equal to 1, it is the only upper bound for 1 and there- 
fore the only possible value of the least upper bound of x and 
1. b) Because x = 1, x is a lower bound for both x and 1 and 
no other lower bound can be greater than x, sox A 1 = x. 
c) Because 0 x x, x is an upper bound for both x and 0 and 
no other bound can be less than x, sox VO = x. d) Because 0 
is the only element less than or equal to 0, it is the only lower 
bound for 0 and therefore the only possible value of the great- 
est lower bound of x and 0. 43. L = (S, C) where S = {@, 
{1}, {2}, {3}, (L 2}, (2, 3}, (1, 2,3}} 45. Yes 47. The 
complement of a subset X C S is its complement S — X. To 
prove this, note that X v (S — X) = land X A(S — X) = 0 
because X u (S — X) = Sand X N (S — X) = Ø. 49. Think 
of the rectangular grid as representing elements in a matrix. 
Thus, we number from top to bottom and within that from left 
to right. The partial order is that (a, b) < (c, d) iff a < c and 
b < d. Note that (1, 1) is the least element under this relation. 
The rules for Chomp as explained in Chapter 1 coincide with 
the rules stated in the preamble here. But now we can identify 
the point (a, b) with the natural number p“~1q*~! for all a and 
b with 1 < a < mand 1 < b < n. This identifies the points 
in the rectangular grid with the set S in this exercise, and the 
partial order < just described is the same as the divides rela- 
tion, because p*!q?-! | p!g! if and only if the exponent 
of p on the left does not exceed the exponent of p on the right, 
and similarly for q. 
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CHAPTER 10 


Section 10.1 


1. a) 


Boston 


Detroit 
Newark 


Washington 


Miami 


Boston 


Detroit 
Newark 


Washington 


Miami 


c) Boston 
Detroit 
Newark 
Washington 
Miami 
d) Boston 
Newark 
Washington 
Miami 
e) Boston 
Detroit 
Newark 
Washington 


Miami 


3. Simple graph 5.Pseudograph 7. Directed graph 
9. Directed multigraph 11. If uRv, then there is an edge 
associated with {u, v}. But {u, v} = {v, u}, so this edge is 
associated with {v, u} and therefore vRu. Thus, by definition, 
R is a symmetric relation. A simple graph does not allow 
loops; therefore, uRu never holds, and so by definition R is 
irreflexive. 
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13. a) A 4 
As £4, A3 
Ag 
b A 4 
As A3 
A4 


SA 


15. Hairy 
Hermit thrush Robin woodpecker 


Mockingbird Blue jay Nuthatch 


17. 
Fermat Descartes 


Mersenne 


e o @ 
Aristotle Euclid Eratosthenes 


i : ) 
Fibonacci Maurolico 


e 
al-Khowarizmi 


Goldbach 


Lamé 


Bézout 


Stirling 


De Morgan Gauss 


Lovelace Dodgson 


Boole 


19 President 
Director, Director, 
R and D marketing 
e 
Director, Chief financial 
operations officer 


Jazz Folk 


No Geanite Boulder a 
LY 


Sand Pebbles 


23 Tigers Blue jays 


Orioles Cardinals 


25. We find the telephone numbers in the call graph for Febru- 
ary that are not present in the call graph for January and vice 
versa. For each number we find, we make a list of the num- 
bers they called or were called by using the edges in the call 
graph. We examine these lists to find new telephone numbers 
in February that had similar calling patterns to defunct tele- 
phone numbers in January. 27. We use the graph model that 
has e-mail addresses as vertices and for each message sent, an 
edge from the e-mail address of the sender to the e-mail ad- 
dress of the recipient. For each e-mail address, we can make 
a list of other addresses they sent messages to and a list of 
other addresses from which they received messages. If two 
e-mail addresses had almost the same pattern, we conclude 
that these addresses might have belonged to the same person 
who had recently changed his or her e-mail address. 29. Let 
V be the set of people at the party. Let E be the set of or- 
dered pairs (u, v) in V X V such that u knows v’s name. The 
edges are directed, but multiple edges are not allowed. Lit- 
erally, there is a loop at each vertex, but for simplicity, the 
model could omit the loops. 31. Vertices are the courses; 
edges are directed; edge uv means that course u is prerequisite 
for course v; courses without prerequisites are vertices with 
in-degree 0; courses that are not prerequisite for any other 
courses are vertices with out-degree 0. 33. Let the set of 
vertices be a set of people, and two vertices are joined by an 
edge if the two people were ever married. Ignoring complica- 
tions, this graph has the property that there are two types of 
vertices (men and women), and every edge joins vertices of 


opposite types. 


37. Represent people in the group by vertices. Put a directed 
edge into the graph for every pair of vertices. Label the edge 
from the vertex representing A to the vertex representing B 
with a + (plus) if A likes B, a — (minus) if A dislikes B, and a 
0 if A is neutral about B. 


Section 10.2 


l.v = 6;e = 6; deg(a) = 2, deg(b) = 4, deg(c) = 1, 
deg(d) = 0, deg(e) = 2, deg(f) = 3; c is pendant; d is isolated. 
3.v = 9; e = 12; deg(a) = 3, deg(b) = 2, deg(c) = 4, 
deg(d) = 0, deg(e) = 6, deg(f) = 0; deg(g) = 4; deg(h) = 2; 
deg(?) = 3; d and fare isolated. 5.No 7.v = 4;e =7; 
deg" (a) = 3, deg-(b) = 1, deg-(c) = 2, deg-(d) = 
deg*t(a) = 1, deg*(b) = 2, deg*(c) = 1,degt(d) =3 9.5 
vertices, 13 edges; deg" (a) = 6, degt(a) = 1, deg(b) = 1 
deg*(b) = 5, deg-(c) = 2, degt(c) = 5, deg"(d) = 4, 
degt (d) = 2, deg (e) = 0, deg*(e) = 0 


13. The number of coauthors that person has; that person’s 
coauthors; a person who has no coauthors; a person who has 
only one coauthor 15. In the directed graph deg’(v) = 
number of calls v received, deg*(v) = number of calls 
v made; in the undirected graph, deg(v) is the number of 
calls either made or received by v. 17. (deg*(v), deg” (v)) 
is the win-loss record of v. 19. In the undirected graph 
model in which the vertices are people in the group and 
two vertices are adjacent if those two people are friends, 
the degree of a vertex is the number of friends in the 
group that person has. By Exercise 18, there are two ver- 
tices with the same degree, which means that there are two 
people in the group with the same number of friends in 
the group. 21. Bipartite 23. Not bipartite 25. Not bi- 
partite 27. a) Parts {h, s, n, w} and {P, Q, R, S}, E = 
{{B n}, {P, w}, {Q, s}, {Qn}, {Rn}, {R, w}, {SA}, {S, s}} 
b) There is. c) {Pw, Qs, Rn, Sh} among others 29. Only 
Barry is willing to marry Uma and Xia. 31. Model this 
with an undirected bipartite graph, with an edge between a 
man and a woman if they are willing to marry each other. By 
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Hall’s theorem, it is enough to show that for every set S of 
women, the set N(S) of men willing to marry them has car- 
dinality at least |S|. Let m be the number of edges between 
S and N(S). Since every vertex in S has degree k, it follows 
that m = k|S|. Because these edges are incident to N(S), it 
follows that m < k|N(S)|. Therefore, k|S| < k|N(S)|, so 
|N(S)| > |S]. 33. Model this with the bipartite graph where 
V, = {(p, i) | pis a winner andi = 1, 2} is the set in which 
each prize winner is represented twice, V, is the set of prizes, 
and an edge represents a player wanting a prize. For any sub- 
set A of V,, N(A) = V5, since every winner wants all of the 
2m prizes. So |N(A)| = 2m = |V,| > |A| and Hall’s theorem 
applies. 35. a) ({a, b, c, f}, {{a b}, {af}, {b, c}, {a f})) 
b) ({a, x, c, f}, {{a, x}, {c x}, {e, x}}) 37.a)n vertices, 
n(n — 1)/2 edges b)n vertices, n edges c)n + 1 vertices, 2n 
edges d) m+n vertices, mn edges e) 2” vertices, n2”! edges 
39. a) 3,3,3,3 b)2,2,2,2 ©)4,3,3,3,3 d)3,3,2,2,2 
e) 3, 3, 3, 3, 3, 3,3,3 41. Each of the n vertices is adjacent 
to each of the other n — 1 vertices, so the degree sequence is 
n—1,n—1,...,n—1(nterms). 


43.7 


45. a) Yes 


b) No c)No d) No 
e) Yes 


° 

f)No 47. First, suppose that d,, d,, ..., d, is graphic. We 
must show that the sequence whose terms are d,—1, d3—1,..., 
dy 41-15 dg, 42> da,+3> +d, 18 graphic once it is put into non- 
increasing order. In Exercise 46 it is proved that if the original 
sequence is graphic, then in fact there is a graph having this 
degree sequence in which the vertex of degree d} is adjacent 
to the vertices of degrees d3, d3,..., dy ,,. Remove from this 
graph the vertex of highest degree (d,). The resulting graph 
has the desired degree sequence. Conversely, suppose that d}, 
d,,..., d, is a nonincreasing sequence such that the sequence 
d, — 1, d; = 1,..., dg 41 — 1, da 4a, da, 43>-++» dy IS graphic 
once it is put into nonincreasing order. Take a graph with this 
latter degree sequence, where vertex v; has degree d; — 1 for 
2 <i<d, +1 and vertex v; has degree d; ford, +2 <i<n. 
Adjoin one new vertex (call it v,), and put in an edge from 
vı to each of the vertices vy, v3,..., Vq,4- The resulting graph 
has degree sequence d,,d5,...,d,. 49. Letd,,d5,...,d, bea 
nonincreasing sequence of nonnegative integers with an even 
sum. Construct a graph as follows: Take vertices v,, v»,..., 
v, and put |d;/2] loops at vertex v,, fori = 1, 2, ..., n. For 
each i, vertex v; now has degree either d; or d; — 1. Because 
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the original sum was even, the number of vertices for which 
deg(v;) = d; — 1 is even. Pair them up arbitrarily, and put in 
an edge joining the vertices in each pair. 51. 17 


53. |a blae o? K a b 
] i c9 @4jc d|c® ; 
de ol de eo” nw de e? 
c® @j\@ @®4|c® d|c® ey 
a b de el de o de ol 
] c® c® c® 
a b de o? a de el 

, ey d ey 
N ; AS de 
c dle @4|c® d|c9 ©] 
e? de e! de el 1 
c® ey Cc 
de ae de e? 
c9 d @j|@ 
e? de e? 
@4|c9® e7 
c® ey 


55. a) Foralln > 1 b) Foralln >3 c)Forn=3_ 4d) For 
alln>0O 57.5 


59. a f b 


G 


c g d 
61. a) The graph with n vertices and no edges b) The disjoint 
union of K,, and K, c) The graph with vertices {v}, ...,v,} 
with an edge between v; and vj unless i = j + 1 (mod n) 
d) The graph whose vertices are represented by bit strings of 
length n with an edge between two vertices if the associated 
bit strings differ in more than one bit 63. v(v — 1)/2 — e 
65.n—1—d,,n—1—-d,_),...,.n-1—d,,n—-1—d, 67. The 
union of G and G contains an edge between each pair of the 
n vertices. Hence, this union is K,,. 
69. Exercise 7: 


Exercise 9: 


Exercise 8: 


71. A directed graph G = (V, E) is its own converse if and only 
if it satisfies the condition (u, v) € E if and only if (v, u) € E. 
But this is precisely the condition that the associated relation 
must satisfy to be symmetric. 


73. P(0,0) P(0,1) P(O, 2) 


75. We can connect P(i, j) and P(k, I) by using |i — k| hops to 
connect P(i, j) and P(k, j) and |j— 1| hops to connect P(k, j) and 
P(k, I). Hence, the total number of hops required to connect 
P(i, j) and P(k, I) does not exceed |i — k| + |j — L|. This is less 
than or equal to m + m = 2m, which is O(m). 
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1. Adjacent 3. Terminal 
Vertex | Vertices Vertex | Vertices 
a b,c, d a a,b,c,d 
b a,d b d 
c a, d č a,b 
d a,b,c d b,c, d 
5.f0 1 1 1 
1 0 0 1 
1 0 0 1 
1 1 1 0 
tt 4 %alo 1 1 1 
0 0 0 1 1041 1 
1 1 0 0 1 10 41 
0O 1 1 1 1 1 1 0 
bfo 1 1 1 1] 9fo 0 1 1 1 
1 0 0 0 0 0 0 1 1 1 
1 0 0 0 0 1 1 0 0 0 
1 0 0 0 0 1 1 0 0 0 
1 0 0 0 0 1 1 0 0 0 
dfo 1 o 1]%fo 1 01 1 
1 0 1 0 1 0 1 0 1 
0O 1 0 1 0 1 O 1 1 
1 O 1 0 1 0i 0 1 
1 1 1 1 0 


Ðo 1 1 0 
1 0 0 1 
i 0 o 4 
0 1 1 0 
100 0 
0 10 0 
00 1 0 
000 1 
11. a b 
15. 102 
01 1 
o- i. 4 
1 2 0 
10 1 0 
01 1 
01 1 
1 0 0 
23. 
b 


25. a)3/7 b) 5/24 


= ON 


oreo 


oreo 


Orroreocoeo 


NF OO 
- OF m. 
ornNOoO 


1 0 0 
0 1 0 
0 0 1 
0 0 0 
O 1 1 
1 0 0 
1 0 0 
O 1 1 

13. 

21. 


1 
1 
1 
0 


uO O m| 


-= 0N 
O - N =. 


c) 19/36 27.a)dense b) sparse 


c) dense d) sparse e) sparse f) sparse 29. Yes 


31. Exercise 13: 


Exercise 14: 


Exercise 15: 


Oe Oo = 


ooor OOs 


0 


O = = 


0 


1 
0 
1 


O.o. COOrFFe 


Boor FOO 


0 

0 

1 

1 

0 0 0 0 

1 0 0 O 

t 1 1 1 

O 1 1 1 

0 0 0 0 0 0 
1 1 1 1 0 0 
O 1 0 0 1 +0 
0 0 1 1 0 1 


33. deg(v) — number of loops at v; deg-(v) 35. 2 if e is not 
a loop, | if e is a loop 


37a), 
1 0 
01 
0 0 
0 0 


1 
0 
0 


— 


— -= © 


ooo 
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by) 0 1 
1 1 0 0 
0 1 0 0 
0 0 1 0 
0 0 1- 1 
c) O O w © Pf Wo a T 
1-0. os O 
B 0 1 0 
Oo ae 4 
where B is the answer to (b) 
d) 
1 1 1 0 0 
0 0 0 1 0 
0 0 0O 0 « 4 
1 0 O 1 + O 
0 1 0 0 - 1 
6 0 Be 1 0 aw 6 


39.Isomorphic 41.Isomorphic 43.Isomorphic 45. Not 
isomorphic 47.Isomorphic 49. G is isomorphic to itself 
by the identity function, so isomorphism is reflexive. Suppose 
that G is isomorphic to H. Then there exists a one-to-one cor- 
respondence f from G to H that preserves adjacency and non- 
adjacency. It follows that f~! is a one-to-one correspondence 
from H to G that preserves adjacency and nonadjacency. 
Hence, isomorphism is symmetric. If G is isomorphic to H 
and H is isomorphic to K, then there are one-to-one corre- 
spondences f and g from G to H and from H to K that pre- 
serve adjacency and nonadjacency. It follows that g of is a 
one-to-one correspondence from G to K that preserves adja- 
cency and nonadjacency. Hence, isomorphism is transitive. 
51. All zeros 53. Label the vertices in order so that all of 
the vertices in the first set of the partition of the vertex set 
come first. Because no edges join vertices in the same set 
of the partition, the matrix has the desired form. 55. C; 
57.n = Sonly 59.4 61.2 63.a) Yes b)No c)No 
65. G = (Vj, E) is isomorphic to H = (V3, E,) if and only 
if there exist functions f from V, to V, and g from E, to E, 
such that each is a one-to-one correspondence and for every 
edge e in E, the endpoints of g(e) are f(v) and f(w) where v 
and w are the endpoints of e. 67. Yes 69. Yes 71. If f 
is an isomorphism from a directed graph G to a directed graph 
H, then f is also an isomorphism from G°”” to H°”". To see 
this note that (u, v) is an edge of G° if and only if (v, u) is 
an edge of G if and only if (f(v), f(u)) is an edge of H if and 
only if (f(u), f(v)) is an edge of H°°””. 73. Many answers 
are possible; for example, C and C} U C3. 75. The product 
is lal where aj is the number of edges from v; to vj when 
i + j and a; is the number of edges incident to v;. 77. The 
graphs in Exercise 45 are a devil’s pair. 
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Section 10.4 


1. a) Path of length 4; not a circuit; not simple b) Not a path 
c) Not a path d) Simple circuit of length 5 3.No 5. No 
7. Maximal sets of people with the property that for any two of 
them, we can find a string of acquaintances that takes us from 
one to the other 9. If a person has Erdős number n, then 
there is a path of length n from that person to Erdős in the col- 
laboration graph, so by definition, that means that that person 
is in the same component as Erdős. If a person is in the same 
component as Erdős, then there is a path from that person to 
Erdős, and the length of the shortest such path is that person’s 
Erdős number. 11. a) Weakly connected b) Weakly con- 
nected c) Not strongly or weakly connected 13. The max- 
imal sets of phone numbers for which it is possible to find 
directed paths between every two different numbers in the 
set 15.a) {a),f}, {ode} b){abhadeh}, {f}, {8} 
c) {a, b, d, e, f, g, h, i}, {c} 17. Suppose the strong compo- 
nents of u and v are not disjoint, say with vertex w in both. 
Suppose x is a vertex in the strong component of u. Then x 
is also in the strong component of v, because there is a path 
from x to v (namely the path from x to u followed by the path 
from u to w followed by the path from w to v) and vice versa. 
Thus, x is in the strong component of v. This shows that the 
strong component of u is a subgraph of the strong component 
of v, and equality follows by symmetry. 19.a)2 b)7 c) 20 
d)61 21. Not isomorphic (G has a triangle; H does not) 
23. Isomorphic (the path wy, uz, Uz, Ug, Us, Uy, U3, Ug, Uy COT- 
responds to the path vj, Vz, V3, V4, Vs, Ver Vz Ve, Vy) 25.a)3 
b)0 ¢)27 d)O 27.a)1 b)O ¢©)2 d)1l e)5 3 29.R 
is reflexive by definition. Assume that (u, v) € R; then there 
is a path from u to v. Then (v, u) € R because there is a path 
from v to u, namely, the path from u to v traversed backward. 
Assume that (u, v) E€ R and (v, w) € R; then there are paths 
from u to v and from v to w. Putting these two paths together 
gives a path from u to w. Hence, (u, w) € R. It follows that 
R is transitive. 3l.c 33.b,c,e,i 35. If a vertex is pen- 
dant it is clearly not a cut vertex. So an endpoint of a cut edge 
that is a cut vertex is not pendant. Removal of a cut edge pro- 
duces a graph with more connected components than in the 
original graph. If an endpoint of a cut edge is not pendant, the 
connected component it is in after the removal of the cut edge 
contains more than just this vertex. Consequently, removal of 
that vertex and all edges incident to it, including the original 
cut edge, produces a graph with more connected components 
than were in the original graph. Hence, an endpoint of a cut 
edge that is not pendant is a cut vertex. 37. Assume there 
exists a connected graph G with at most one vertex that is 
not a cut vertex. Define the distance between the vertices u 
and v, denoted by d(u, v), to be the length of the shortest path 
between u and v in G. Let s and ¢ be vertices in G such that 
d(s, t) is a maximum. Either s or t (or both) is a cut vertex, 
so without loss of generality suppose that s is a cut vertex. 
Let w belong to the connected component that does not con- 
tain t of the graph obtained by deleting s and all edges inci- 
dent to s from G. Because every path from w to ¢ contains s, 


d(w, t) > d(s, t), which is a contradiction. 39. a) Denver- 
Chicago, Boston—New York b) Seattle—Portland, Portland- 
San Francisco, Salt Lake City-Denver, New York—Boston, 
Boston—Burlington, Boston—Bangor 41. A minimal set of 
people who collectively influence everyone (directly or in- 
directly); {Deborah} 43. An edge cannot connect two ver- 
tices in different connected components. Because there are at 
most C(n;, 2) edges in the connected component with n; ver- 
tices, it follows that there are at most 5a C(n,, 2) edges in 
the graph. 45. Suppose that G is not connected. Then it has 
a component of k vertices for some k, 1 < k < n-— 1. 
The most edges G could have is C(k, 2) + C(n — k, 2) = 
[k(k -l)+(n-K(n—k — 1)]/2 = K — nk + (r? — n)/2. 
This quadratic function of f is minimized at k = n/2 and 
maximized at k = lork = n — 1. Hence, if G is not 
connected, the number of edges does not exceed the value of 
this function at 1 and at n — 1, namely, (n — 1)(n — 2)/2. 
47.a)l b)2 c)6 d)21 49.a) Removing an edge from 
a cycle leaves a path, which is still connected. b) Removing 
an edge from the cycle portion of the wheel leaves that por- 
tion still connected and the central vertex still connected to 
it as well. Removing a spoke leaves the cycle intact and the 
central vertex still connected to it as well. c) Any four ver- 
tices, two from each part of the bipartition, are connected 
by a 4-cycle; removing one edge does not disconnect them. 
d) Deleting the edge joining (b4, by, ..., bi- 0, b -> Bn) 
and (bi, bọ ... , bip l, bip --- > ba) does not dis- 
connect the graph because these two vertices are still 


joined via the path (b,, b} ... , bi 0, bio -~ 0), 
(bi by, ss a Dii O Bur eco Ds (Bp By esa By, L 
bap -7 1), (Dy bo .., bip L bip -~ 0) ifn < 2 and 
b, = 0, and similarly in the other three cases. 51. If G 


is complete, then removing vertices one by one leaves a com- 
plete graph at each step, so we never get a disconnected graph. 
Conversely, if edge uv is missing from G, then removing 
all the vertices except u and v creates a disconnected graph. 
53. Both equal min(m, n). 55. Let G be a graph with n ver- 
tices; then x(G) < n— 1. Let C be a smallest edge cut, leaving 
a nonempty proper subset S of the vertices of G disconnected 
from the complementary set S’ = V — S. If xy is an edge of G 
for every x € Sand y € S’, then the size of C is |S||S’|, which 
is at least n — 1, so x(G) < A(G). Otherwise, let x € S and 
y E S' be nonadjacent vertices. Let T consist of all neighbors 
of x in S’ together with all vertices of $ — {x} with neigh- 
bors in S’. Then T is a vertex cut, because it separates x and y. 
Now look at the edges from x to T N S’ and one edge from 
each vertex of T N S to S’; this gives us |T| distinct edges that 
lie in C, so A(G) = |C| > |T| > «K(G). 57.2 59. Let 
the simple paths P, and P, be u = xo x),...,x, = v and 
U = Yo Yi <--> Ym = V, respectively. The paths thus start out at 
the same vertex. Since the paths do not contain the same set 
of edges, they must diverge eventually. If they diverge only 
after one of them has ended, then the rest of the other path is 
a simple circuit from v to v. Otherwise we can suppose that 
Xo = Yo X1 = Yi -> X; = Yp but x4, Æ Yi To form 
our simple circuit, we follow the path y,, y;,;, ¥j49, and so on, 


until it once again first encounters a vertex on P} (possibly as 
early as y;,,, no later than y,,). Once we are back on P,, we 
follow it along—forwards or backwards, as necessary—to re- 
turn to x;. Since x; = y;, this certainly forms a circuit. It must 
be a simple circuit, since no edge among the x;s or the y;s 
can be repeated (P, and P, are simple by hypothesis) and no 
edge among the x,s can equal one of the edges y, that we used, 
since we abandoned P, for P} as soon as we hit P,;. 61. The 
graph G is connected if and only if every off-diagonal entry of 
A +A? +A? +- +A”! is positive, where A is the adjacency 
matrix of G. 63. If the graph is bipartite, say with parts A 
and B, then the vertices in every path must alternately lie in 
A and B. Therefore, a path that starts in A, say, will end in B 
after an odd number of steps and in A after an even number 
of steps. Because a circuit ends at the same vertex where it 
starts, the length must be even. Conversely, suppose that all 
circuits have even length; we must show that the graph is bi- 
partite. We can assume that the graph is connected, because 
if it is not, then we can just work on one component at a time. 
Let v be a vertex of the graph, and let A be the set of all ver- 
tices to which there is a path of odd length starting at v, and 
let B be the set of all vertices to which there is a path of even 
length starting at v. Because the component is connected, ev- 
ery vertex lies in A or B. No vertex can lie in both A and B, 
because if one did, then following the odd-length path from v 
to that vertex and then back along the even-length path from 
that vertex to v would produce an odd circuit, contrary to the 
hypothesis. Thus, the set of vertices has been partitioned into 
two sets. To show that every edge has endpoints in different 
parts, suppose that xy is an edge, where x € A. Then the 
odd-length path from v to x followed by xy produces an even- 
length path from v to y, soy € B. (Similarly, if x € B.) 
65. (H, W,H,W,(boat), Ø — (H,W,, H,W,(boat)) > 
(H,H,W,\(boat), W,) — (W, H,W,H,(boat)) > 
(H,W,(boat), H,W,) > (Ø, H, W, H, W, (boat)) 


Section 10.5 


1. Neither 3. No Euler circuit; a, e, c, e, b, e, d, b, a, c, d 
5.a, b,c, d,c, e, d, b,e,a,e,a 7.a, i, h, 2, d, e, f, 8, c, e, h, d, 
c,a,b,i,c,b,h,a 9. No, A still has odd degree. 11. When 
the graph in which vertices represent intersections and edges 
streets has an Euler path 13. Yes 15.No 17. If there is 
an Euler path, then as we follow it each vertex except the 
starting and ending vertices must have equal in-degree and 
out-degree, because whenever we come to a vertex along an 
edge, we leave it along another edge. The starting vertex must 
have out-degree 1 larger than its in-degree, because we use 
one edge leading out of this vertex and whenever we visit 
it again we use one edge leading into it and one leaving it. 
Similarly, the ending vertex must have in-degree 1 greater 
than its out-degree. Because the Euler path with directions 
erased produces a path between any two vertices, in the under- 
lying undirected graph, the graph is weakly connected. Con- 
versely, suppose the graph meets the degree conditions stated. 
If we add one more edge from the vertex of deficient out- 
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degree to the vertex of deficient in-degree, then the graph 
has every vertex with equal in-degree and out-degree. Be- 
cause the graph is still weakly connected, by Exercise 16 this 
new graph has an Euler circuit. Now delete the added edge to 
obtain the Euler path. 19. Neither 21. No Euler circuit; 
a, d,e, d, b,a,e,c,e,b,c,b,e 23. Neither 25. Follow the 
same procedure as Algorithm 1, taking care to follow the 
directions of edges. 27. a)n = 2 b)None c) None 
d)n = 1 29. Exercise 1:1 time; Exercises 2-7: 0 times 
31.a, b, c, d, e, a is a Hamilton circuit. 33. No Hamilton 
circuit exists, because once a purported circuit has reached e 
it would have nowhere to go. 35. No Hamilton circuit ex- 
ists, because every edge in the graph is incident to a vertex 
of degree 2 and therefore must be in the circuit. 37. a, b, 
c, f, d, e is a Hamilton path. 39. f, e, d, a, b, c is a Hamil- 
ton path. 41. No Hamilton path exists. There are eight ver- 
tices of degree 2, and only two of them can be end vertices 
of a path. For each of the other six, their two incident edges 
must be in the path. It is not hard to see that if there is to be a 
Hamilton path, exactly one of the inside corner vertices must 
be an end, and that this is impossible. 43. a, b, c, f, i, h, 
g, d, e is a Hamilton path. 45.m =n > 2 47.a) (i) No, 
(ii) No, (iii) Yes b) (i) No, (ii) No, (iii) Yes c) (i) Yes, (ii) Yes, 
(iii) Yes d) (i) Yes, (ii) Yes, (iii) Yes 49. The result is triv- 


ial for n = 1: code is 0, 1. Assume we have a Gray code 
of order n. Let cy, ..., Cp k = 2” be such a code. Then 
Oc, ..., OC Icy, ..., le, is a Gray code of order n + 1. 
51. procedure Fleury(G = (V, E): connected multigraph 
with the degrees of all vertices even, V = {v,,..., v,,}) 
v:=v] 
circuit := v 
H:=G 
while H has edges 


e := first edge with endpoint v in H (with respect to 
listing of V) such that e is not a cut edge of H, if 
one exists, and simply the first edge in H with 
endpoint v otherwise 
w := other endpoint of e 
circuit := circuit with e, w added 
vi=w 
H:=H-e 

return circuit {circuit is an Euler circuit} 


53. If G has an Euler circuit, then it also has an Euler path. 
If not, add an edge between the two vertices of odd degree 
and apply the algorithm to get an Euler circuit. Then delete 
the new edge. 55. Suppose G = (V, E) is a bipartite graph 
with V = V, U V;, where V, N V, = Ø and no edge 
connects a vertex in V, and a vertex in V,. Suppose that G 
has a Hamilton circuit. Such a circuit must be of the form 
a, by, Ay, bo, ..., ap by, aj, Where a; E V, and b; € V, for 
i= 1,2, ..., k. Because the Hamilton circuit visits each vertex 
exactly once, except for v,, where it begins and ends, the num- 
ber of vertices in the graph equals 2k, an even number. Hence, 
a bipartite graph with an odd number of vertices cannot have 
a Hamilton circuit. 
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9 10 11 12 


A knight’s tour can be made by following the moves 8, 10, 1, 
7,9, 2, 11, 5, 3, 12, 6,4. 61. We represent the squares of a 
4 x 4 chessboard as follows: 


uua 


There are only two moves from each of the four corner 
squares. If we include all the edges 1—10, 1-7, 16-10, and 16- 
7, a circuit is completed too soon, so at least one of these edges 
must be missing. Without loss of generality, assume the path 
starts 1-10, 10-16, 16-7. Now the only moves from square 
3 are to squares 5, 10, and 12, and square 10 already has two 
incident edges. Therefore, 3—5 and 3—12 must be in the Hamil- 
ton circuit. Similarly, edges 8—2 and 8-15 must be in the cir- 
cuit. Now the only moves from square 9 are to squares 2, 7, 
and 15. If there were edges from square 9 to both squares 2 and 
15, a circuit would be completed too soon. Therefore, the edge 
9-7 must be in the circuit giving square 7 its full complement 
of edges. But now square 14 is forced to be joined to squares 
5 and 12, completing a circuit too soon (S—14—12-3-5). This 
contradiction shows that there is no knight’s tour on the 4 x 4 
board. 63. Because there are mn squares on an mX n board, 
if both m and n are odd, there are an odd number of squares. 
Because by Exercise 62 the corresponding graph is bipartite, 
by Exercise 55 it has no Hamilton circuit. Hence, there is no 
reentrant knight’s tour. 65. a) If G does not have a Hamil- 
ton circuit, continue as long as possible adding missing edges 
one at a time in such a way that we do not obtain a graph 
with a Hamilton circuit. This cannot go on forever, because 
once we’ve formed the complete graph by adding all miss- 
ing edges, there is a Hamilton circuit. Whenever the process 
stops, we have obtained a (necessarily noncomplete) graph H 


with the desired property. b) Add one more edge to H. This 
produces a Hamilton circuit, which uses the added edge. The 
path consisting of this circuit with the added edge omitted 
is a Hamilton path in H. cœ) Clearly v; and v, are not adja- 
cent in H, because H has no Hamilton circuit. Therefore, they 
are not adjacent in G. But the hypothesis was that the sum of 
the degrees of vertices not adjacent in G was at least n. This 
inequality can be rewritten as n — deg(v,) < deg(v,). But 
n — deg(y,,) is just the number of vertices not adjacent to v,,. 
d) Because there is no vertex following v, in the Hamilton 
path, v, is not in S. Each one of the deg(v,) vertices adjacent 
to v, gives rise to an element of S, so S contains deg(v, ) ver- 
tices. e) By part (c) there are at most deg(v,)— 1 vertices other 
than v, not adjacent to v,,, and by part (d) there are deg(v, ) ver- 
tices in S, none of which is v,. Therefore, at least one vertex 
of S is adjacent to v,. By definition, if v, is this vertex, then 
H contains edges v,v, and v,v,,,;, where 1 < k < n—1. 
Ê) Now vy, Va, <., Vets Vio Ym Yn- <--> Yk Vy iS a Hamil- 
ton circuit in H, contradicting the construction of H. There- 
fore, our assumption that G did not originally have a Hamilton 
circuit is wrong, and our proof by contradiction is complete. 
67. Assume that there is a Hamilton circuit H. First assume 
that edge aj ¢ H. Then ab, ai, jk, and jl are forced to be in 
H, which means that kl ¢ H, and then gk and dl are in H. If 
ch were not in H, then bc, cd, gh, and hi would all have to be 
in H, completing a circuit without all the vertices. It follows 
that ch € H. A circuit would be completed too soon if either 
both be and cd were in H, or both gh and hi were in H, so by 
symmetry WOLOG assume that cd € H and hi € H and the 
other two edges are not. This then forces bf € H and fg € H, 
and we have completed a circuit without vertex e. Going back 
to the beginning, we now know that aj € H. Because of the 
120° rotational symmetry of the figure, it then follows that dl 
and gk are in H as well. It is clearly now impossible to include 
all three of the vertices i, j, and k in the Hamilton circuit, and 
our proof by contradiction is complete. 


Section 10.6 


1. a) Vertices are the stops, edges join adjacent stops, weights 
are the times required to travel between adjacent stops. 
b) Same as part (a), except weights are distances between ad- 
jacent stops. c) Same as part (a), except weights are fares be- 
tween stops. 3. 16 5. Exercise 2: a, b, e, d, z, Exercise 3: a, c, 
d, e, g, z, Exercise 4: a, b, e, h, l,m, p,s,z  7.a)ac,db)a,c, 
df c)c,d,f e)b,d,e,g,z 9.a)Direct b) Via New York 
c) Via Atlanta and Chicago d) Via New York 11. a) Via 
Chicago b) Via Chicago c) Via Los Angeles d) Via 
Chicago 13. a) Via Chicago b) Via Chicago c) Via Los 
Angeles d) Via Chicago 15. Do not stop the algorithm 
when z is added to the set S$. 17. a) Via Woodbridge, via 
Woodbridge and Camden b) Via Woodbridge, via Wood- 
bridge and Camden 19. For instance, sightseeing tours, 
street cleaning 


21. 


— 
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23. O(n?) 25. a-c-b-d-a (or the same circuit starting at 
some other point and/or traversing the vertices in reverse 
order) 27. San Francisco—Denver—Detroit-New York—Los 
Angeles—San Francisco (or the same circuit starting at some 
other point and/or traversing the vertices in reverse order) 
29. Consider this graph: 


The circuit a-b-a-c-a visits each vertex at least once (and the 
vertex a twice) and has total weight 6. Every Hamilton cir- 
cuit has total weight 103. 31. Let v4, vy, ..., v, be a topo- 
logical ordering of the vertices of the given directed acyclic 
graph. Let w(i j) be the weight of edge v;v,. Iteratively define 
P(i) with the intent that it will be the weight of a longest path 
ending at v; and C(i) with the intent that it will be the vertex 
preceding v; in some longest path: For i from 1 to n, let P(i) 
be the maximum of P(j) + w(j, i) over all j < i such that v;v; 
is an edge in the directed graph (and if such a j exists let C(i) 
be a value of j for which this maximum is achieved) and let 
P(i) = O if there are no such values of j. At the conclusion 
of this loop, a longest path can be found by choosing i that 
maximizes P(i) and following the C links back to the start of 
the path. 


Section 10.7 
1. Yes 3. b 5. No 
a d 
° 
7. Yes a d 
c 
b 


9.No 11. A triangle is formed by the planar representation 
of the subgraph of K; consisting of the edges connecting v}, 
Vj, and v3. The vertex v, must be placed either within the tri- 
angle or outside of it. We will consider only the case when 
v4 is inside the triangle; the other case is similar. Drawing the 
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three edges from v,, v,, and v3 to v4 forms four regions. No 
matter which of these four regions v; is in, it is possible to join 
it to only three, and not all four, of the other vertices. 13.8 
15. Because there are no loops or multiple edges and no sim- 
ple circuits of length 3, and the degree of the unbounded re- 
gion is at least 4, each region has degree at least 4. Thus, 
2e > 4r,orr < e/2. Butr = e — v + 2, so we have 
e—v+2 < e/2, which implies that e < 2v — 4. 17. Asin 
the argument in the proof of Corollary 1, we have 2e > 5r 
andr = e — v + 2. Thus,e —v+2 < 2e/5, which 


implies that e < (5/3) — (10/3). 19. Only (a) and (c) 
21. Not homeomorphic to K33; 23. Planar 25. Nonplanar 
27.a) 1 b)3c¢)9d)2e)4f)16 29. Draw K,, n as described 
in the hint. The number of crossings is four times the number 
in the first quadrant. The vertices on the x-axis to the right 
of the origin are (1, 0), (2, 0), ... , (m/2, 0) and the vertices 
on the y-axis above the origin are (0, 1), (0, 2), ..., (0, 2/2). 
We obtain all crossings by choosing any two numbers a and 
b with 1 < a < b < m/2 and two numbers r and s 
with 1 < r < s < n/2; we get exactly one crossing in 
the graph between the edge connecting (a, 0) and (0, s) and 


the edge connecting (b, 0) and (0, r). Hence, the number of 
crossings in the first quadrant is C (£ 2) -C (£ 2) = 


one : a Hence, the total number of cross- 
ings is 4 - mn(m — 2)(n — 2)/64 = mn(m — 2)(n — 2)/16. 
3l.a)2 b)2 c)2 d)2 e)2 f)2 33. The formula is valid 
forn < 4.Ifn > 4, by Exercise 32 the thickness of K, is at 
least C(n, 2)/(3n — 6) = (n+ 1 + 2 )/6 rounded up. Be- 
cause this quantity is never an integer, it equals |(n + 7)/6]. 
35. This follows from Exercise 34 because K,,, , has mn edges 


m,n 


and m +n vertices and has no triangles because it is bipartite. 


RIA 


Section 10.8 


3. Three colors 
B A 


2a, w o 


D 


5 3 7.3 92 11.3 13.Graphs with no edges 
15. 3 if nis even, 4ifnisodd 17. Period 1: Math 115, Math 
185; period 2: Math 116, CS 473; period 3: Math 195, CS 101; 
period 4: CS 102; period 5: CS 273 19.5 21. Exercise 5: 
3 Exercise 6:6 Exercise 7: 3 Exercise 8: 4 Exercise 9: 3 
Exercise 10:6 Exercise 11:4 23. a) 2 if n is even, 3 if n is 
odd b)n 25. Two edges that have the same color share no 


1. Four colors 
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endpoints. Therefore, if more than n/2 edges were colored the 
same, the graph would have more than 2(n/2) = n vertices. 
27.5 29. Color 1: e, f, d, color 2: c, a, i, g, color 3: h, b, j 
31. Color Cg; 33. Four colors are needed to color W,, when 
n is an odd integer greater than 1, because three colors are 
needed for the rim (see Example 4), and the center vertex, be- 
ing adjacent to all the rim vertices, will require a fourth color. 
To see that the graph obtained from W, by deleting one edge 
can be colored with three colors, consider two cases. If we 
remove a rim edge, then we can color the rim with two colors, 
by starting at an endpoint of the removed edge and using the 
colors alternately around the portion of the rim that remains. 
The third color is then assigned to the center vertex. If we 
remove a spoke edge, then we can color the rim by assigning 
color #1 to the rim endpoint of the removed edge and colors 
#2 and #3 alternately to the remaining vertices on the rim, 
and then assign color #1 to the center. 35. Suppose that G 
is chromatically k-critical but has a vertex v of degree k — 2 
or less. Remove from G one of the edges incident to v. By 
definition of “k-critical,’ the resulting graph can be colored 
with k — 1 colors. Now restore the missing edge and use this 
coloring for all vertices except v. Because we had a proper 
coloring of the smaller graph, no two adjacent vertices have 
the same color. Furthermore, v has at most k— 2 neighbors, so 
we can color v with an unused color to obtain a proper (k— 1)- 
coloring of G. This contradicts the fact that G has chromatic 
number k. Therefore, our assumption was wrong, and every 
vertex of G must have degree at least k — 1. 37.a)6 b)7 
c)9 d)11 39. Represent frequencies by colors and zones 
by vertices. Join two vertices with an edge if the zones these 
vertices represent interfere with one another. Then a k-tuple 
coloring is precisely an assignment of frequencies that avoids 
interference. 41. We use induction on the number of ver- 
tices of the graph. Every graph with five or fewer vertices can 
be colored with five or fewer colors, because each vertex can 
get a different color. That takes care of the basis case(s). So 
we assume that all graphs with k vertices can be 5-colored 
and consider a graph G with k + 1 vertices. By Corollary 2 in 
Section 10.7, G has a vertex v with degree at most 5. Remove 
v to form the graph G’. Because G” has only k vertices, we 
5-color it by the inductive hypothesis. If the neighbors of v 
do not use all five colors, then we can 5-color G by assigning 
to v acolor not used by any of its neighbors. The difficulty 
arises if v has five neighbors, and each has a different color in 
the 5-coloring of G’. Suppose that the neighbors of v, when 
considered in clockwise order around v, are a, b, c, m, and p. 
(This order is determined by the clockwise order of the curves 
representing the edges incident to v.) Suppose that the colors 
of the neighbors are azure, blue, chartreuse, magenta, and 
purple, respectively. Consider the azure-chartreuse subgraph 
(i.e., the vertices in G colored azure or chartreuse and all the 
edges between them). If a and c are not in the same compo- 
nent of this graph, then in the component containing a we 
can interchange these two colors (make the azure vertices 
chartreuse and vice versa), and G’ will still be properly col- 
ored. That makes a chartreuse, so we can now color v azure, 


and G has been properly colored. If a and c are in the same 
component, then there is a path of vertices alternately colored 
azure and chartreuse joining a and c. This path together with 
edges av and vc divides the plane into two regions, with b in 
one of them and m in the other. If we now interchange blue 
and magenta on all the vertices in the same region as b, we 
will still have a proper coloring of G’, but now blue is avail- 
able for v. In this case, too, we have found a proper coloring 
of G. This completes the inductive step, and the theorem is 
proved. 43. We follow the hint. Because the measures of 
the interior angles of a pentagon total 540°, there cannot be 
as many as three interior angles of measure more than 180° 
(reflex angles). If there are no reflex angles, then the pen- 
tagon is convex, and a guard placed at any vertex can see all 
points. If there is one reflex angle, then the pentagon must 
look essentially like figure (a) below, and a guard at vertex v 
can see all points. If there are two reflex angles, then they 
can be adjacent or nonadjacent (figures (b) and (c)); in either 
case, a guard at vertex v can see all points. [In figure (c), 
choose the reflex vertex closer to the bottom side.] Thus, for 
all pentagons, one guard suffices, so g(5) = 1. 


R 
(a) (b) 


(c) 


45. The figure suggested in the hint (generalized to have k 
prongs for any k > 1) has 3k vertices. The sets of locations 
from which the tips of different prongs are visible are dis- 
joint. Therefore, a separate guard is needed for each of the 
k prongs, so at least k guards are needed. This shows that 
g(3k) > k = |3k/3].Ifn = 3k + i, where 0 < i < 2, 
then g(n) > g(3k) > k = |n/3]. 


Supplementary Exercises 


1. 2500 3.Yes 5.Yes 7.J;;-; n; vertices, D nin; 
edges 9.a)Ifx € N(A U B), then x is adjacent to some 
vertex v E AUB. WOLOG suppose v € A; then x € N(A) and 
therefore also in N(A)UN(B). Conversely, if x € N(A)UN(B), 
then WOLOG suppose x € N(A). Thus, x is adjacent to 
some vertexv € A C AUB,sox E€ N(A U B). b)If 
x € N(A N B), then x is adjacent to some vertex v € A N B. 
Since both v € A and v € B, it follows that x € N(A) and 
x € N(B), whence x € N(A) N N(B). For the counterexample, 
let G = ({u, v, w}, {{u v}, {v,w}}), A = {u}, and B = {w}. 
11. (c, a, p, x, n, m) and many others 13. (c, d, a, b) and 
many others 15. 6 times the number of triangles divided by 
the number of paths of length2 17. a) The probability that 


two actors each of whom has appeared in a film with a ran- 
domly chosen actor have appeared in a film together b) The 
probability that two of a randomly chosen person’s Facebook 
friends are themselves Facebook friends c) The probability 
that two of a randomly chosen person’s coauthors are them- 
selves coauthors d) The probability that two proteins that 
each interact with a randomly chosen protein interact with 
each other e) The probability that two routers each of which 
has a communications link to a randomly chosen router are 
themselves linked 19. Complete subgraphs containing the 
following sets of vertices: {b, c, e, f}, {a, b, g}, {a d, g}, 
{d, e, g}, {b, e, g} 21. Complete subgraphs containing the 
following sets of vertices: {b, c, d, j,k}, {a, b, j, k}, {ef g, i}, 
{a, b, i}, {aij}, {b, d, e}, {b, e, i}, {b, i j}, {ahi}, {h, i j} 
23. {c, d} is a minimum dominating set. 

25. a) 


NG 


27,.a)1 b)2 c)3 29.a) A path from u to v ina graph G 
induces a path from f(u) to f(v) in an isomorphic graph H. 
b) Suppose f is an isomorphism from G to H. If vo, Vp ..., 
V,» Vo is a Hamilton circuit in G, then f(v9), f1) --., fp), 
f(vVo) must be a Hamilton circuit in H because it is still a cir- 
cuit and f(v;) # f~; forO < i <j <n. ©) Suppose f is 
an isomorphism from G to H. If vo, Vi, ..., Vp» Vo is an Euler 
circuit in G, then f(v), f(v,), .-., FfOn) fp) must be an Eu- 
ler circuit in H because it is a circuit that contains each edge 
exactly once. d) Two isomorphic graphs must have the same 
crossing number because they can be drawn exactly the same 
way in the plane. e) Suppose f is an isomorphism from G 
to H. Then v is isolated in G if and only if f(v) is isolated in 
H. Hence, the graphs must have the same number of isolated 
vertices. f) Suppose f is an isomorphism from G to H. If G 
is bipartite, then the vertex set of G can be partitioned into 
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V; and V, with no edge connecting vertices within V} or ver- 
tices within V,. Then the vertex set of H can be partitioned 
into f(V,) and f(V,) with no edge connecting vertices within 
(Vi) or vertices within f(V,). 31.3  33.a) Yes b) No 
35.No 37. Yes 39. Ife is acut edge with endpoints u and 
v, then if we direct e from u to v, there will be no path in the 
directed graph from v to u, or else e would not have been a 
cut edge. Similar reasoning works if we direct e from v to u. 
4l.n — 1 43. Let the vertices represent the chickens. We 
include the edge (u, v) in the graph if and only if chicken u 
dominates chicken v. 45. By the handshaking theorem, the 
average vertex degree is 2m/n, which equals the minimum de- 
gree; it follows that all the vertex degrees are equal. 47. K3 3 
and the skeleton of a triangular prism 49. a) A Hamilton 
circuit in the graph exactly corresponds to a seating of the 
knights at the Round Table such that adjacent knights are 
friends. b) The degree of each vertex in this graph is at least 
2n—1-—(n—1) =n > (2n/2), so by Dirac’s theorem, this graph 
has a Hamilton circuit. c)a,b,d,fg,z 5Sl.a)4 b)2 ¢)3 
d)4 e)4 f)2 53. a) Suppose that G = (V, E). Leta, b E V. 
We must show that the distance between a and b in G is at 
most 2. If {a, b} ¢ E this distance is 1, so assume {a, b} E E. 
Because the diameter of G is greater than 3, there are vertices 
u and v such that the distance in G between u and v is greater 
than 3. Either u or v, or both, is not in the set {a, b}. Assume 
that u is different from both a and b. Either {a, u} or {b, u} 
belongs to E; otherwise a, u, b would be a path in Gof length 
2. So, without loss of generality, assume {a, u} € E. Thus, v 
cannot be a or b, and by the same reasoning either {a, v} € E 
or {b, v} € E. In either case, this gives a path of length less 
than or equal to 3 from u to v in G, a contradiction. b) Suppose 
G = (V, E). Let a, b E V. We must show that the distance be- 
tween a and b in G does not exceed 3. If {a, b} & E, the result 
follows, so assume that {a, b} € E. Because the diameter of G 
is greater than or equal to 3, there exist vertices u and v such 
that the distance in G between u and v is greater than or equal 
to 3. Either u or v, or both, is not in the set {a, b}. Assume u is 
different from both a and b. Either {a, u} E E or {b, u} E€ E; 
otherwise a, u, b is a path of length 2 in G. So, without loss 
of generality, assume {a, u} € E. Thus, v is different from a 
and from b. If {a, v} E€ E, then u, a, v is a path of length 2 
in G, so {a, v} É E and thus {b, v} € E (or else there would 
be a path a, v, b of length 2 in G). Hence, {u, b} ¢ E; other- 
wise u, b, v is a path of length 2 in G. Thus, a, v, u, b is a path 
of length 3 in G, as desired. 55.a, b,e,z 57.a,c,d,f, 8,z 
59. If G is planar, then because e < 3v — 6, G has at most 27 
edges. (If G is not connected it has even fewer edges.) Simi- 
larly, G has at most 27 edges. But the union of G and Gis Ki, 
which has 55 edges, and 55 > 27 +27. 61. Suppose that G 
is colored with k colors and has independence number i. Be- 
cause each color class must be an independent set, each color 
class has no more than i elements. Thus, there are at most ki 
vertices. 63. a) C(n, m)p”(1 — p)"™™ b) np c) To generate 
a labeled graph G, as we apply the process to pairs of ver- 
tices, the random number x chosen must be less than or equal 
to 1/2 when G has an edge between that pair of vertices and 
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greater than 1/2 when G has no edge there. Hence, the prob- 
ability of making the correct choice is 1/2 for each edge and 
1/2°%) overall. Hence, all labeled graphs are equally likely. 
65. Suppose P is monotone increasing. If the property of not 
having P were not retained whenever edges are removed from 
a simple graph, there would be a simple graph G not having P 
and another simple graph G’ with the same vertices but with 
some of the edges of G missing that has P. But P is monotone 
increasing, so because G’ has P, so does G obtained by adding 
edges to G’. This is a contradiction. The proof of the converse 


is similar. 


CHAPTER 11 
Section 11.1 


1. (a), (c), (e) 3.a)a b)a,b,c,d,f,h,j,q,t Oe, g,i, k,l, 
m,n, 0,p,r,s,u d)q,r e)c Ê p g)f,b,a h)e,f,l,m,n 
5.No 7. Level 0: a; level 1: b, c, d; level 2: e through k (in 
alphabetical order); level 3: / through r; level 4: s, t; level 5: u 
9. a) The entire tree b) c, g, h, o, p and the four edges cg, ch, 
ho,hpc)ealone 11.a)1lb)2 13.a)3b)9 15. a) The 
“only if” part is Theorem 2 and the definition of a tree. Sup- 
pose G is a connected simple graph with n vertices and n — 1 
edges. If G is not a tree, it contains, by Exercise 14, an edge 
whose removal produces a graph G’, which is still connected. 
If G’ is not a tree, remove an edge to produce a connected 
graph G”. Repeat this procedure until the result is a tree. This 
requires at most n — 1 steps because there are only n — 1 edges. 
By Theorem 2, the resulting graph has n — 1 edges because 
it has n vertices. It follows that no edges were deleted, so G 
was already a tree. b) Suppose that G is a tree. By part (a), 
G has n — | edges, and by definition, G has no simple cir- 
cuits. Conversely, suppose that G has no simple circuits and 
has n — 1 edges. Let c equal the number of components of G, 
each of which is necessarily a tree, say with n; vertices, where 
yi, n; = n. By part (a), the total number of edges in G is 
È; (a; - 1) = n-—c. Since we are given that this equals n— 1, 
it follows that c = 1, i.e., G is connected and therefore satis- 
fies the definition of a tree. 17.9999 19.2000 21.999 
23. 1,000,000 dollars 25. No such tree exists by Theorem 4 


because it is impossible for m = 2 or m = 84. 


27. Complete binary tree of height 4: 


Complete 3-ary tree of height 3: 


29. a) By Theorem 3 it follows thatn = mi + 1. Because 
i+] =n, we have l = n—i, sol = (mi+1)—i = (m—1)i+1. b) We 
haven = mi + 1 and i + l = n. Hence, i = n — l. It follows that 
n = m(n—l)+1. Solving for n gives n = (ml—1)/(m—1). From 
i = n—l we obtain i = [(ml— 1)/(m— 1)]—1 = (l—1)/(m— 1). 
3l.n—t 33.a)l b)3 c)5 35.a) The parent directory 
b) A subdirectory or contained file ¢) A subdirectory or con- 
tained file in the same parent directory d) All directories in 
the path name e) All subdirectories and files continued in the 
directory or a subdirectory of this directory, and so on f) The 
length of the path to this directory or file g) The depth of the 
system, i.e., the length of the longest path 37. Let n = 2%, 
where k is a positive integer. If k = 1, there is nothing to 
prove because we can add two numbers with n — 1 = 1 pro- 
cessor in log2 = 1 step. Assume we can add n = 2 numbers 
in log n steps using a tree-connected network of n — 1 pro- 
cessors. Let x], X2, ..., X, be 2n = 2**! numbers that we 
wish to add. The tree-connected network of 2n — | processors 
consists of the tree-connected network of n — 1 processors 
together with two new processors as children of each leaf. In 
one step we can use the leaves of the larger network to find 
Xi +X, X3 + Xy <--> X2n-1 + Xp Giving us n numbers, which, 
by the inductive hypothesis, we can add in log n steps using 
the rest of the network. Because we have used logn + 1 steps 
and log(2n) = log2 + logn = 1 + logn, this completes the 
proof. 39.conly 4l.candh 43. Suppose a tree T has 
at least two centers. Let u and v be distinct centers, both with 
eccentricity e, with u and v not adjacent. Because T is con- 
nected, there is a simple path P from u to v. Let c be any other 
vertex on this path. Because the eccentricity of c is at least e, 
there is a vertex w such that the unique simple path from c to 
w has length at least e. Clearly, this path cannot contain both u 
and v or else there would be a simple circuit. In fact, this path 
from c to w leaves P and does not return to P once it, possibly, 
follows part of P toward either u or v. Without loss of gener- 
ality, assume this path does not follow P toward u. Then the 
path from u to c to w is simple and of length more than e, a 
contradiction. Hence, u and v are adjacent. Now because any 
two centers are adjacent, if there were more than two centers, 
T would contain K;, a simple circuit, as a subgraph, which is 
a contradiction. 


45. T D T3 T4 
OA £ 
Ts Ts 
Tı 


47. The statement is that every tree with n vertices has a path 
of length n — 1, and it was shown only that there exists a tree 
with n vertices having a path of length n — 1. 


Section 11.2 


1. banana 


papaya 
3. a)3 b)1 c)4 d)5 
5. the 


dog 


lazy 


7. At least [log, 4] = 2 weighings are needed, because there 
are only four outcomes (because it is not required to determine 
whether the coin is lighter or heavier). In fact, two weighings 
suffice. Begin by weighing coin | against coin 2. If they bal- 
ance, weigh coin | against coin 3. If coin 1 and coin 3 are the 
same weight, coin 4 is the counterfeit coin, and if they are not 
the same weight, then coin 3 is the counterfeit coin. If coin 1 
and coin 2 are not the same weight, again weigh coin | against 
coin 3. If they balance, coin 2 is the counterfeit coin; if they 
do not balance, coin | is the counterfeit coin. 9. At least 
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[log, 13] = 3 weighings are needed. In fact, three weighings 
suffice. Start by putting coins 1, 2, and 3 on the left-hand side 
of the balance and coins 4, 5, and 6 on the right-hand side. 
If equal, apply Example 3 to coins 1, 2, 7, 8, 9, 10, 11, and 
12. If unequal, apply Example 3 to 1, 2, 3, 4, 5, 6, 7, and 8. 
11. The least number is five. Call the elements a, b, c, and d. 
First compare a and b; then compare c and d. Without loss 
of generality, assume thata < bandc < d. Next compare 
a and c. Whichever is smaller is the smallest element of the 
set. Again without loss of generality, suppose a < c. Finally, 
compare b with both c and d to completely determine the or- 
dering. 13. The first two steps are shown in the text. After 
22 has been identified as the second largest element, we re- 
place the leaf 22 by —oo in the tree and recalculate the winner 
in the path from the leaf where 22 used to be up to the root. 
Next, we see that 17 is the third largest element, so we re- 
peat the process: replace the leaf 17 by —oo and recalculate. 
Next, we see that 14 is the fourth largest element, so we re- 
peat the process: replace the leaf 14 by —oo and recalculate. 
Next, we see that 11 is the fifth largest element, so we repeat 
the process: replace the leaf 11 by —co and recalculate. The 
process continues in this manner. We determine that 9 is the 
sixth largest element, 8 is the seventh largest element, and 3 
is the eighth largest element. The trees produced in all steps, 
except the second to last, are shown here. 
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15. The value of a vertex is the list element currently there, 
and the label is the name (i.e., location) of the leaf responsible 
for that value. 
procedure tournament sort(d,, ..., ap) 
k := [logn] 
build a binary tree of height k 
for i := 1 ton 
set the value of the ith leaf to be a; and its label to 
be itself 
fori := n+ 1 to 2% 
set the value of the ith leaf to be —oo and its label to 
be itself 
for i := k — 1 downto 0 
for each vertex v at level i 
set the value of v to the larger of the values of its 
children and its label to be the label of the child 
with the larger value 
for i := 1 ton 
c; ‘= value at the root 
let v be the label of the root 
set the value of v to be — o0 
while the label at the root is still v 
v := parent(v) 
set the value of v to the larger of the values of its 
children and its label to be the label of the child 
with the larger value 
{ci ---, ¢, is the list in nonincreasing order} 
17. k — 1, where n = 2% 19.a) Yes b)No c¢)Yes d) Yes 
21. a: 000, e: 001, i: 01, k: 1100, o: 1101, p: 11110, w: 11111 
23.a: 11; b: 101; c: 100; d: 01; e: 00; 2.25 bits (Note: This 
coding depends on how ties are broken, but the average num- 
ber of bits is always the same.) 25. There are four possible 
answers in all, the one shown here and three more obtained 
from this one by swapping t and v and/or swapping u and w. 


27. A:0001; B:101001; C:11001; D:00000; E:100; 
F:001100; G:001101; H:0101; 1:0100; J:110100101; 
K:1101000; L:00001; M:10101; N:0110; 0:0010; P:101000; 
Q:1101001000; R:1011; S:0111; T:111; U:00111; V:110101; 
W:11000; X:11010011; Y:11011; Z:1101001001 29. A:2; 
E:1; N:010; R:011; T:02;Z:00 31.n 33. Because the tree 
is rather large, we have indicated in some places to “see text.” 
Refer to Figure 9; the subtree rooted at these square or circle 
vertices is exactly the same as the corresponding subtree in 
Figure 9. First player wins. 


+1 
32 max 


min 
see 
text 
+1 


21 11 1 1 2 | max 


1 3 
-1 see see -l -1 +1 
text text 
+1 +1 
© min 
+1 -1 +1 
1 
1 


35. a)$1 b)$3 c)—-$3 37. See the figures shown next. 
a)0 b)O c)1 db) This position cannot have occurred in a 
game; this picture is impossible. 


0 0 
O|X| xX XIOIX 
a) xloļo b) o|x|x 
X O 
a a ar d 
O|X|X O/;}X|X XIOIX XIOJX 
xIiOoJO xXIOJO O|X|X OIXĪX 
O x O| X OJo O O 
a lo al lo 
OIXIX O;}X]X XIOIX X/O}X 
XIOJO XJOJIO OIX|X O/;}X|X 
OIXIX X |O/}X x|O;O OJIXJO 
draw draw wins draw 


+1 
X O 
k ojo 
X X 
e lg ea 
xX|X|O x O X O 
OJO O;O;}X OJO 
X X x X XI|XIX 
oP N ie X wins 
XIXIO XIXJO XJOJO X O 
ojoo OJO OlOoIX o0OlļlOIX 
X X XIOIX X X X|O|X 
O wins ol ail al 
XIXJO XJOJO XI|XIO 
OJO|X O;O;X OJOIX 
XIOIX - XIXIX XIjO}X 
draw X wins draw 


39. Proof by strong induction: Basis step: When there are 
n = 2 stones in each pile, if first player takes two stones from 
a pile, then second player takes one stone from the remain- 
ing pile and wins. If first player takes one stone from a pile, 
then second player takes two stones from the other pile and 
wins. Inductive step: Assume inductive hypothesis that sec- 
ond player can always win if the game starts with two piles 
of j stones for all 2 < j < k, where k > 2, and con- 
sider a game with two piles containing k + 1 stones each. 
If first player takes all the stones from one of the piles, then 
second player takes all but one stone from the remaining pile 
and wins. If first player takes all but one stone from one of the 
piles, then second player takes all the stones from the other 
pile and wins. Otherwise first player leaves j stones in one pile, 
where 2 < j < k, and k + 1 stones in the other pile. Second 
player takes the same number of stones from the larger pile, 
also leaving j stones there. At this point the game consists of 
two piles of j stones each. By the inductive hypothesis, the 
second player in that game, who is also the second player in 
our actual game, can win, and the proof by strong induction is 
complete. 41.7;49 43. Value of tree is 1. Note: The sec- 
ond and third trees are the subtrees of the two children of the 
root in the first tree whose subtrees are not shown because of 
space limitations. They should be thought of as spliced into 
the first picture. 
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+1 -1 


X X xjo X| Q X x x 
x [0] [0] 
[0] 
Fs 6-5=1 5-5=0 4-5=-1 6-5=1 5-5=0 
ie) fe) 
x x 


X X XJO X X 
(9) (9) 
O (9) 
4-6=-2 6-6=0 5-6=-1 6-6=0 5-6=-1 
Section 11.3 
1. 0 
1 
ra 


1.1 1.2 
0<1<11<12<2<3 


12.41 
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5.No 7.a, b,d, e, f, g,c 9.a, b, e, k, l, m, f, 8, n, r, 
s6 d h oij pa d, b, i ée m, j, n 0 a f, è €k, 
h,p,l 13.d,f,g,e,b,c,a 15. k, l, m, e, f, r, $, n, 8, b, 
c,o, h,i, p,q,j,d,a 


17. a) + 


oN Â 


b) ++x * xy/xy, +x/+ * xyxy €) xxy * +xy/+, xxy * x+y/+ 
d) (Œ + (& * y)) + @/y)), Œ + (Œ * y) +.x)/y)) 


19. a) = 


ras 


b) -NABUA-BA c) ABAABA-— uU- 


d) (A N B) — (Au (B —- A))) 21.14 23.a)1 b)1 o)4 
d) 2205 


27. Use mathematical induction. The result is trivial for a list 
with one element. Assume the result is true for a list with n 
elements. For the inductive step, start at the end. Find the se- 
quence of vertices at the end of the list starting with the last 


leaf, ending with the root, each vertex being the last child of 
the one following it. Remove this leaf and apply the inductive 
hypothesis. 29. c, d, b, f, g, h, e, a in each case 31. Proof 
by mathematical induction. Let S(X) and O(X) represent the 
number of symbols and number of operators in the well- 
formed formula X, respectively. The statement is true for well- 
formed formulae of length 1, because they have 1 symbol 
and 0 operators. Assume the statement is true for all well- 
formed formulae of length less than n. A well-formed for- 
mula of length n must be of the form *XY, where * is an 
operator and X and Y are well-formed formulae of length 
less than n. Then by the inductive hypothesis S(#XY) = 
S(X) + S(Y) = [O(X) + 1] + [0(Y) + 1] = O(X) + OY) + 2. 
Because O(#XY) = 1 + O(X) + O(Y), it follows that 
S(*XY) = O(#XY) + 1. 33. xy+zx0+Xx0, xyz ++ yxt4, 
Xyxyooxyoozo+, XZX, zz+0, yyyyooo, zx-+yz+o, for instance 


Section 11.4 


l. m-n+1 


pa b c 5. 
g f e 
a) <> b) 


NNN 
wed 


d) e) f) 
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9, 4 b SA g a b f g a b Y g a b f g 
c d e h c d e h c d e h c d e h 
a b f g a b f g a b f g a b f g 
e e e e o e 
© | > | e | | | e | | @ © | | | 
c d e h c d e h È d e h c d e h 
a b f g a b f g a b f g a b f g 

o e @ o @ o 
| © © | | © | | | © | @ | o | | 
g d e h c d e h a d e h f d e h 
a b f g a b f g b f 


a b f g 
le d e h 
11. a)3 b) 16 œ©)4 d)5 


B ¢ 
p SS 


e h i 
€ d 
f & j 
a 
be e 
ce f 
d 


17. a) A path of length 6 b) A path of length 5 c) A path 
of length 6 d) Depends on order chosen to visit the vertices; 
may be a path of length7 19. With breadth-first search, the 
initial vertex is the middle vertex, and the n spokes are added 
to the tree as this vertex is processed. Thus, the resulting tree 
is Kı „n. With depth-first search, we start at the vertex in the 
middle of the wheel and visit a neighbor—one of the vertices 
on the rim. From there we move to an adjacent vertex on the 
rim, and so on all the way around until we have reached every 
vertex. Thus, the resulting spanning tree is a path of length n. 
21. With breadth-first search, we fan out from a vertex of de- 
gree m to all the vertices of degree n as the first step. Next, 
a vertex of degree n is processed, and the edges from it to 
all the remaining vertices of degree m are added. The result 
is a K,,_, anda K,,,_, with their centers joined by an edge. 
With depth-first search, we travel back and forth from one par- 
tite set to the other until we can go no further. If m = n or 
m =n- 1, then we get a path of length m + n — 1. Otherwise, 
the path ends while some vertices in the larger partite set have 
not been visited, so we back up one link in the path to a vertex 
v and then successively visit the remaining vertices in that set 
from v. The result is a path with extra pendant edges coming 


fo 
a 
a 
> 
9 


15. 


n m 


œa @——e@n 
> @—!__®% 


out of one end of the path. 23. A possible set of flights 
to discontinue are: Boston—New York, Detroit—Boston, 
Boston—Washington, New York—Washington, New York- 
Chicago, Atlanta—Washington, Atlanta—Dallas, Atlanta—Los 
Angeles, Atlanta—St. Louis, St. Louis—Dallas, St. Louis— 
Detroit, St. Louis—Denver, Dallas-San Diego, Dallas—Los 
Angeles, Dallas—San Francisco, San Diego—Los Angeles, Los 
Angeles—San Francisco, San Francisco—Seattle. 25. Proof 
by induction on the length of the path: If the path has length 0, 
then the result is trivial. If the length is 1, then u is adjacent to 
v, so u is at level 1 in the breadth-first spanning tree. Assume 
that the result is true for paths of length /. If the length of a 
path is /+ 1, let u’ be the next-to-last vertex in a shortest path 
from v to u. By the inductive hypothesis, u’ is at level Z in the 
breadth-first spanning tree. If u were at a level not exceeding 
L, then clearly the length of the shortest path from v to u would 
also not exceed /. So u has not been added to the breadth-first 
spanning tree yet after the vertices of level / have been added. 
Because u is adjacent to w’, it will be added at level / + 1 (al- 
though the edge connecting u’ and u is not necessarily added). 
27. a) No solution 


29. Start at a vertex and proceed along a path without repeat- 
ing vertices as long as possible, allowing the return to the start 
after all vertices have been visited. When it is impossible to 


S-76 Answers to Odd-Numbered Exercises 


continue along a path, backtrack and try another extension of 
the current path. 31. Take the union of the spanning trees 
of the connected components of G. They are disjoint, so the 
result is a forest. 33.m—n+c 35. Assume that we wish 
to find the length of a shortest path from v, to every other ver- 
tex of G using Algorithm 2. In line 2 of that algorithm, add 
L(v,) := 0, and add the following as a third step in the then 
clause at the end: L(w) := 1 + L(v). 37. Add an instruction 
to the BFS algorithm to mark each vertex as it is encoun- 
tered. When BFS terminates we have found (all the vertices 
of) one component of the graph. Repeat, starting at an un- 
marked vertex, and continue in this way until all vertices 
have been marked. 39. Trees 41. Use depth-first search 
on each component. 43. If an edge uv is not followed while 
we are processing vertex u during the depth-first search pro- 
cess, then it must be the case that the vertex v had already 
been visited. There are two cases. If vertex v was visited af- 
ter we started processing u, then, because we are not finished 
processing u yet, v must appear in the subtree rooted at u (and 
hence, must be a descendant of u). On the other hand, if the 
processing of v had already begun before we started process- 
ing u, then why wasn’t this edge followed at that time? It must 
be that we had not finished processing v, in other words, that 
we are still forming the subtree rooted at v, so u is a descen- 
dant of v, and hence, v is an ancestor of u. 45. Certainly 
these two procedures produce the identical spanning trees if 
the graph we are working with is a tree itself, because in this 
case there is only one spanning tree (the whole graph). This 
is the only case in which that happens, however. If the origi- 
nal graph has any other edges, then by Exercise 43 they must 
be back edges and hence, join a vertex to an ancestor or de- 
scendant, whereas by Exercise 34, they must connect vertices 
at the same level or at levels that differ by 1. Clearly these 
two possibilities are mutually exclusive. Therefore, there can 
be no edges other than tree edges if the two spanning trees 
are to be the same. 47. Because the edges not in the span- 
ning tree are not followed in the process, we can ignore them. 
Thus, we can assume that the graph was a rooted tree to begin 
with. The basis step is trivial (there is only one vertex), so we 
assume the inductive hypothesis that breadth-first search ap- 
plied to trees with n vertices have their vertices visited in order 
of their level in the tree and consider a tree T with n + 1 ver- 
tices. The last vertex to be visited during breadth-first search 
of this tree, say v, is the one that was added last to the list of 
vertices waiting to be processed. It was added when its par- 
ent, say u, was being processed. We must show that v is at 
the lowest (bottom-most, i.e., numerically greatest) level of 
the tree. Suppose not; say vertex x, whose parent is vertex w, 
is at a lower level. Then w is at a lower level than u. Clearly 
v must be a leaf, because any child of v could not have been 
seen before v is seen. Consider the tree T” obtained from T by 
deleting v. By the inductive hypothesis, the vertices in T’ must 
be processed in order of their level in T’ (which is the same as 
their level in T, and the absence of v in T’ has no effect on the 
rest of the algorithm). Therefore, u must have been processed 
before w, and therefore v would have joined the waiting list 


before x did, a contradiction. Therefore, v is at the bottom- 
most level of the tree, and the proof is complete. 49. We 
modify the pseudocode given in Algorithm 2 by initializing 
m to be 0 at the beginning of the algorithm, and adding the 
statements “m := m + 1” and “assign m to vertex v” after the 
statement that removes vertex v from L. 51. If a directed 
edge uv is not followed while we are processing its tail u dur- 
ing the depth-first search process, then it must be the case that 
its head v had already been visited. There are three cases. If 
vertex v was visited after we started processing u, then, be- 
cause we are not finished processing u yet, v must appear in the 
subtree rooted at u (and hence, must be a descendant of u), so 
we have a forward edge. Otherwise, the processing of v must 
have already begun before we started processing u. If it had 
not yet finished (i.e., we are still forming the subtree rooted 
at v), then u is a descendant of v, and hence, v is an ances- 
tor of u (we have a back edge). Finally, if the processing of v 
had already finished, then by definition we have a cross edge. 
53. Let T be the spanning tree constructed in Figure 3 and 
T,, T,, T;, and T, the spanning trees in Figure 4. Denote by 
d(T’, T") the distance between T’ and T”. Then d(T, T,) = 6, 
d(T, T>) = 4, d(T, T3) = 4, d(T, T4) = 2, d(T, T3) = 4, 
d(T, T3) = 4, d(T, Ty) = 6, d(T, T3) = 4, d(Ty, T4) = 2, 
and d(T}, T,) = 4. 55. Suppose e) = {u, v} is as speci- 
fied. Then T, U {e,} contains a simple circuit C containing 
e,. The graph T, — {e,} has two connected components; the 
endpoints of e, are in different components. Travel C from 
u in the direction opposite to e; until you come to the first 
vertex in the same component as v. The edge just crossed is 
e. Clearly, T, U {e,;} — {ey} is a tree, because e, was on C. 
Also T, — {e;} U {e3} is a tree, because e, reunited the two 
components. 


57. Exercise 18: 
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Exercise 21: 


59. First construct an Euler circuit in the directed graph. Then 
delete from this circuit every edge that goes to a vertex pre- 
viously visited. 61. According to Exercise 60, a directed 
graph contains a circuit if and only if there are any back edges. 
We can detect back edges as follows. Add a marker on each 
vertex v to indicate what its status is: not yet seen (the ini- 
tial situation), seen (i.e., put into T) but not yet finished (i.e., 
visit(v) has not yet terminated), or finished (i.e., visit(v) has 
terminated). A few extra lines in Algorithm 1 will accomplish 
this bookkeeping. Then to determine whether a directed graph 


has a circuit, we just have to check when looking at edge uv 
whether the status of v is “seen.” If that ever happens, then we 
know there is a circuit; if not, then there is no circuit. 


Section 11.5 


1. Deep Springs—Oasis, Oasis—Dyer, Oasis—Silver Peak, Sil- 
ver Peak—Goldfield, Lida—Gold Point, Gold Point—Beatty, 
Lida—Goldfield, Goldfield-Tonopah, Tonopah—Manhattan, 
Tonopah—Warm Springs 3. {e, f}, {c, f}, {e, h}, {h i}, 
{b, c}, {b, d}, {a, d}, {g, h} 


5 


Atlanta 


7. {ef}, {a, d}, {h,i}, {b, d}, {of}, {e, h}, {b, c}, {8, h} 
9. a 


11. Instead of choosing minimum-weight edges at each stage, 
choose maximum-weight edges at each stage with the same 
properties. 


13. a b 
3 
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e 
Cc d 


17. First find a minimum spanning tree T of the graph G with 
n edges. Then fori = 1 to n — 1, delete only the ith edge of 
T from G and find a minimum spanning tree of the remaining 
graph. Pick the one of these n—1 trees with the shortest length. 
19. If all edges have different weights, then a contradiction is 
obtained in the proof that Prim’s algorithm works when an 
edge e,,, is added to T and an edge e is deleted, instead of 
possibly producing another spanning tree. 


2. 4 2 b3 c 1 d 


@ 
3 1 2 
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23. Same as Kruskal’s algorithm, except start with T := this 
set of edges and iterate from i = 1 toi = n — 1 — s, where s is 
the number of edges you start with. 
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27. By Exercise 24, at each stage of Sollin’s algorithm a forest 
results. Hence, after n — 1 edges are chosen, a tree results. It 
remains to show that this tree is a minimum spanning tree. Let 
T be a minimum spanning tree with as many edges in com- 
mon with Sollin’s tree S as possible. If T # S, then there is an 
edge e € S — T added at some stage in the algorithm, where 
prior to that stage all edges in S are also in T. TU {e} contains 
a unique simple circuit. Find an edge e’ € S — T and an edge 
e" € T — S on this circuit and “adjacent” when viewing the 
trees of this stage as “‘supervertices.” Then by the algorithm, 
w(e’) < w(e’’). So replace T by T — {e’’} U {e’} to produce 
a minimum spanning tree closer to S than T was. 29. Each 
of the r trees is joined to at least one other tree by a new edge. 
Hence, there are at most r/2 trees in the result (each new tree 
contains two or more old trees). To accomplish this, we need 
to add r — (r/2) = r/2 edges. Because the number of edges 
added is integral, it is at least [r/2]. 31. If k > logn, then 
n/2* < 1, so [n/2*] = 1, so by Exercise 30 the algorithm 
is finished after at most log n iterations. 33. Suppose that a 
minimum spanning tree T contains edge e = uv that is the 
maximum weight edge in simple circuit C. Delete e from T. 
This creates a forest with two components, one containing u 
and the other containing v. Follow the edges of the path C—e, 
starting at u. At some point this path must jump from the com- 
ponent of T—e containing u to the component of T—e contain- 
ing v, say using edge f. This edge cannot be in T, because e can 
be the only edge of T joining the two components (otherwise 
there would be a simple circuit in T). Because e is the edge 
of greatest weight in C, the weight of f is smaller. The tree 
formed by replacing e by f in T therefore has smaller weight, 
acontradiction. 35. The reverse-delete algorithm must ter- 
minate and produce a spanning tree, because the algorithm 
never disconnects the graph and upon termination there can 
be no more simple circuits. The edge deleted at each stage of 
the algorithm must have been the edge of maximum weight in 
whatever circuits it was a part of. Therefore, by Exercise 33 
it cannot be in any minimum spanning tree. Since only edges 
that could not have been in any minimum spanning tree have 
been deleted, the result must be a minimum spanning tree. 


Supplementary Exercises 


1. Suppose T is a tree. Then clearly T has no simple circuits. 
If we add an edge e connecting two nonadjacent vertices u and 
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v, then obviously a simple circuit is formed, because when e 
is added to T the resulting graph has too many edges to be a 
tree. The only simple circuit formed is made up of the edge e 
together with the unique path in T from v to u. Suppose T sat- 
isfies the given conditions. All that is needed is to show that T 
is connected, because there are no simple circuits in the graph. 
Assume that T is not connected. Then let u and v be in sepa- 
rate connected components. Adding e = {u, v} does not sat- 
isfy the conditions. 3. Suppose that a tree T has n vertices 
of degrees dj, dy, ..., d,, respectively. Because 2e = pe d 
ande = n — 1, we have 2(n — 1) = };_ d; Because each 
d; > 1, it follows that 2(n — 1) = n+ };_;(d; — 1), or that 
n—2 = };_(d; — 1). Hence, at most n — 2 of the terms of 
this sum can be 1 or more. Hence, at least two of them are 0. 
It follows that d; = 1 for at least two values of i. 5.2n — 2 
7. A tree has no circuits, so it cannot have a subgraph homeo- 
morphic to K; 3 or Ks. 9. Color each connected component 
separately. For each of these connected components, first root 
the tree, then color all vertices at even levels red and all ver- 
tices at odd levels blue. 11. Upper bound: k"; lower bound: 
2[k/2]|"! 

13. 


PARA gift 


15. Because B,,, is formed from two copies of B,, one shifted 
down one level, the height increases by 1 as k increases by 1. 
Because By had height 0, it follows by induction that B, has 
heightk. 17. Because the root of B,,, is the root of B, with 
one additional child (namely the root of the other B,), the de- 
gree of the root increases by 1 as k increases by 1. Because By 
had a root with degree 0, it follows by induction that B, has a 
root with degree k. 


S4 


21. Use mathematical induction. The result is trivial for k = 0. 
Suppose it is true for k— 1. T,_, is the parent tree for T. By in- 
duction, the child tree for T can be obtained from To, ..., T,_» 
in the manner stated. The final connection of 7,_, to 7,_ is as 
stated in the definition of S,-tree. 


23. procedure /evel(T: ordered rooted tree with root r) 
queue := sequence consisting of just the root r 
while queue contains at least one term 

v := first vertex in queue 

list v 

remove v from queue and put children of v onto 
the end of queue 


25. Build the tree by inserting a root for the address 0, and 
then inserting a subtree for each vertex labeled i, for i a posi- 
tive integer, built up from subtrees for each vertex labeled ij 
for j a positive integer, and soon. 27.a) Yes b) No c) Yes 
29. The resulting graph has no edge that is in more than one 
simple circuit of the type described. Hence, it is a cactus. 


d e f 
35. a) 1 2 3 ye T E j 
Eag 3 5 
“TT 


37.6 39. a) Ofor 00, 11 for 01, 100 for 10, 101 for 11 (exact 
coding depends on how ties were broken, but all versions are 
equivalent); 0.645n for string of length n b) 0 for 000, 100 for 
001, 101 for 010, 110 for 100, 11100 for 011, 11101 for 101, 
11110 for 110, 11111 for 111 (exact coding depends on how 
ties were broken, but all versions are equivalent); 0.5326n 
for string of length n 41. Let G’ be the graph obtained by 
deleting from G the vertex v and all edges incident to v. A 
minimum spanning tree of G can be obtained by taking an 
edge of minimal weight incident to v together with a mini- 
mum spanning tree of G’. 43. Suppose that edge e is the 
edge of least weight incident to vertex v, and suppose that T is 
a spanning tree that does not include e. Add e to T, and delete 
from the simple circuit formed thereby the other edge of the 
circuit that contains v. The result will be a spanning tree of 
strictly smaller weight (because the deleted edge has weight 
greater than the weight of e). This is a contradiction, so T 
must include e. 45. Because paths in trees are unique, an 
arborescence T of a directed graph G is just a subgraph of G 
that is a tree rooted at r, containing all the vertices of G, with 
all the edges directed away from the root. Thus, the in-degree 
of each vertex other than r is 1. For the converse, it is enough 
to show that for each v € V there is a unique directed path 
from r to v. Because the in-degree of each vertex other than 
r is 1, we can follow the edges of T backwards from v. This 
path can never return to a previously visited vertex, because 
that would create a simple circuit. Therefore, the path must 


eventually stop, and it can stop only at r, whose in-degree 
is not necessarily 1. Following this path forward gives the 
path from r to v required by the definition of arborescence. 
47. a) Run the breadth-first search algorithm, starting from v 
and respecting the directions of the edges, marking each ver- 
tex encountered as reachable. b) Running breadth-first search 
on G°, again starting at v, respecting the directions of the 
edges, and marking each vertex encountered, will identify all 
the vertices from which v is reachable. c) Choose a vertex 
vı and using parts (a) and (b) find the strong component con- 
taining v,, namely all vertices w such that w is reachable from 
vı and vy, is reachable from w. Then choose another vertex 
v, not yet in a strong component and find the strong compo- 
nent of v). Repeat until all vertices have been included. The 
correctness of this algorithm follows from the definition of 
strong component and Exercise 17 in Section 10.4. 


CHAPTER 12 


Section 12.1 
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c) 110 111 d) 
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000 001 000 


9. (0, 0) and (1, 1) 
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27.a) True, as a table of values can show b) False; take x = 1, 
y = 1,z = 1, for instance c) False; take x = 1, y = 1, 
z = 0, for instance 29. By De Morgan’s laws, the comple- 
ment of an expression is like the dual except that the comple- 
ment of each variable has been taken. 31.16 33. If we 
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replace each 0 by F, 1 by T, Boolean sum by v, Boolean 
product by A, and by ~ (and x by p and y by q so that the 
variables look like they represent propositions, and the equals 
sign by the logical equivalence symbol), then xy = x + y 
becomes ~(p A q) = =p V nq and x+y = xy becomes 
~a(p Vq) = =p Ang. 35. By the domination, distributive, 
and identity laws, x Vx =Vx)AL=(QVxX)AQVX) =XV 
(x Ax) = x V0 = x. Similarly, xAx=(X Ax) VO=(XA 
XVXAX) =A QVI =xXAIL =x. 37. Because 
Ov 1 = land0A 1 = 0 by the identity and commutative 
laws, it follows that 0=1. Similarly, because 1 Vv 0 = 1 and 
1A0 = 1, it follows that 1=0. 39. First, note that x \0 = 0 
and x V 1 = 1 for all x, as can easily be proved. To prove the 
first identity, it is sufficient to show that (x V y V(XA y) =1 
and (x V y) A(x A y) = 0. By the associative, commutative, 
distributive, domination, and identity laws, (x V y) V (x A y) = 
y VIev@ Ay=yVI@VN AVY) = yvi 
yyl =yV@vy=(yVyVx = 1Vx = 1 and 
(XV VYJAGAY)=YALXAGVY)=HVAIAANDVAAYI=YA 
[(OV@A y)] =VYAGAyY) =X A (y A y) = XA0=0. The 
second identity is proved ina similar way. 41. Using the hy- 
potheses, Exercise 35, and the distributive law it follows that 
x=xVOH=xXVAEVY HF AVAYVY=HXVY = 0. 
Similarly, y = 0. To prove the second statement, note that 
X=XAL=XAAKAY)=(KXAXAY=XAy= l. Similarly, 
y = 1. 43. Use Exercises 39 and 41 in the Supplementary 
Exercises in Chapter 9 and the definition of a complemented, 
distributed lattice to establish the five pairs of laws in the def- 
inition. 
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lo a)xyz b)xyz e)z d)xyz 3.a)xyz + xyz + 
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c) xyz + xyz + xyz + xyz d)xyz + xyz 5.wxyz + 
wxyz + wxyz + wxyz + Wxy Z + WX yz + WxXyz + wxyZz 
7.a) x+y+z b)x+y+z c)x+y+z 9. yi +y + +y, =0 
if and only if y; = 0 for i = 1, 2, ..., n. This holds if and only 
if x; = 0 when y; = x; and x; = 1 when y; = x;. 11. a)x+y+z 
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19. It is impossible to represent 


x using + and - because there is no way to get the value 0 if 
the input is 1. 
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Section 12.4 


1.a) y y b) xy and xy 
X 
x 1 
3. a) y y 
x 1 
# 
b) y y c) y y 
x 1 x 1 1 
z 1 x| 1 1 
5.a) yz yz yz yz 
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b) xyz, Xy Z, XYZ 
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7. a) yz yz yz yz 13. a) yz yz yz yz 


Wx 


b) yz yz Jz yz 


x Wx 1 
X 1 1 
b) wxyz, W XYZ, WXY Z, WXYZ 
c) yw Z KK K 
x i 1 15.a) 
X3X4X5 X3X4X5 X3X4X5 X3X4X5 X3X4X5 X3X4X5 X3X4X5 X3X4X%5 
z l ! X2 1 1 
Xiz 
9. Implicants: xyz, Xyz, X Y Z, XYZ, XY, XZ, yZ, prime implicants: 
xy, XZ, yz; essential prime implicants: xy, xz, yz ne 
yz yz YE yz ae 
x 1 1 1 
x 1 
b) 
X3X4X5 X3X4X5 X3X4X5 A3X 4X5 X3X4X5 X3X4X5 XzX 4X5 X3X 4X5 
: : Han 
11. The 3-cube on the right corresponds to w; the 3-cube given 
by the top surface of the whole figure represents x; the 3-cube 5 
: 2 
given by the back surface of the whole figure represents y; the 
3-cube given by the right surfaces of both the left and the right o 
3-cube represents z. In each case, the opposite 3-face repre- "a d ! 
sents the complemented literal. The 2-cube that represents wz 7 
is the right face of the 3-cube on the right; the 2-cube that eee) d 1 
represents xy is bottom rear; the 2-cube that represents y Z is 
front left. 
c) 
X3X4X5 X3X4X5 X3X4X5 X3X4X5 X3X4X5 X3X4X5 X3X4X5 X3X4%5 
i2 1 1 1 1 
X 1X9 
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17.a)64 b)6 19. Rows 1 and 4 are considered adjacent. 
The pairs of columns considered adjacent are: columns 1 and 
4, l and 12, 1 and 16, 2 and 11, 2 and 15, 3 and 6, 3 and 10, 4 
and 9, 5 and 8, 5 and 16, 6 and 15, 7 and 10, 7 and 14, 8 and 
13, 9 and 12, 11 and 14, 13 and 16. 
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Burton 


23.a)xz b)y c)xz+xz+yz d)xz+xy+yz 25.a) wxzt+ 
wxy + wyz + wxyz b) xyz + w yz + wxyz + wxyz + W Xyz 
c) yz +wxy + wxy+wxyz d)wy + yz + xy + wxz + wXxz 
27. x(y +z) 
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31.xz +xz 33. We use induction on n. If n = 1, then we 
are looking at a line segment, labeled 0 at one end and 1 at 
the other end. The only possible value of k is also 1, and if the 
literal is x}, then the subcube we have is the 0-dimensional 
subcube consisting of the endpoint labeled 1, and if the lit- 
eral is x,, then the subcube we have is the 0-dimensional sub- 
cube consisting of the endpoint labeled 0. Now assume that 
the statement is true for n; we must show that it is true for 
n + 1. If the literal x,,,; (or its complement) is not part of 
the product, then by the inductive hypothesis, the product 
when viewed in the setting of n variables corresponds to an 
(n — k)-dimensional subcube of the n-dimensional cube, and 
the Cartesian product of that subcube with the line segment 
[0, 1] gives us a subcube one dimension higher in our given 
(n+ 1)-dimensional cube, namely having dimension (n+1)—k, 
as desired. On the other hand, if the literal x, ,, (or its com- 
plement) is part of the product, then the product of the re- 
maining k — | literals corresponds to a subcube of dimension 
n — (k — 1) = (n+ 1) — k in the n-dimensional cube, and 
that slice, at either the 1-end or the 0-end in the last variable, 
is the desired subcube. 


Supplementary Exercises 


Lax=0,y=0,z=0;x=1,y=1,z 
y=0,z=0;x=0,y=0,z=1x=0,y 1,z=0; 
x ly=0,z Lx 1, y LOK lly Í, 
z= 1 ¢) Novalues 3.a) Yes b) No c) No d) Yes 5.2?” 
7.a) If Fi, xX) = 1, then (F + G)y,...,x,) = 
F(x, 3 X) + GO, ...,%,) = 1 by the dominance law. 
Hence, F < F + G. Db)If (FG), ...,x,) = 1, then 
F(X, e03 Xn) © GX ...,x,) = 1. Hence, F(x, ...,x,) = 1. It 
follows that FG < F. 9. Because F(x), ...,x,) = 1 implies 
that F(x, ..., x,) = 1, < is reflexive. Suppose that F < G and 
G < F. Then F(x, ...,x,) = 1 if and only if G@, ...,x,) = 1. 
This implies that F = G. Hence, < is antisymmetric. Suppose 
that F < G < H. Then if F(x, ...,x,) = 1, it follows that 
G(X ---, Xa) = 1, which implies that H(x, ...,x,) = 1. 
Hence, F < H, so < is transitive. 11. a)x= 1, y=0,z=0 
b)x=Ly=0z=0 c)x=1,y=0,z=0 


1 b)x=0, 
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15. Yes, as a truth table shows 17. a)6 b)5 c)5 d)6 


19. x 
y 
|} cms 


21. x3 + Xx, 23. Suppose it were with weights a and b. 
Then there would be a real number T such that xa + yb > T 
for (1,0) and (0,1), but with xa + yb < T for (0,0) and (1,1). 
Hence, a > T.b>T,0<T,anda+b < T. Thus, a and b are 
positive, which implies that a + b > a > T, a contradiction. 


CHAPTER 13 


Section 13.1 


1. a) sentence = noun phrase intransitive verb phrase 
= article adjective noun intransitive verb phrase = 
article adjective noun intransitive verb =... 
(after 3 steps) ... = the happy hare runs. 

b) sentence = noun phrase intransitive verb phrase 
= article adjective noun intransitive verb phrase 
= article adjective noun intransitive verb 
adverb... (after 4 steps). .. => the sleepy tortoise runs 
quickly 

c) sentence = noun phrase transitive verb phrase 
noun phrase > article noun transitive verb phrase 
noun phrase > article noun transitive verb noun 
phrase = article noun transitive verb article 
noun =... (after 4 steps)...= the tortoise passes the hare 
d) sentence = noun phrase transitive verb phrase 
noun phrase = article adjective noun transitive 
verb phrase noun phrase = article adjective noun 
transitive verb noun phrase = article adjective 
noun transitive verb article adjective noun 
=... (after 6 steps)...= the sleepy hare passes the happy 
tortoise 


3. The only way to get a noun, such as tortoise, at the end is to 
have a noun phrase at the end, which can be achieved only via 
the production sentence — noun phrase transitive verb 
phrase noun phrase. However, transitive verb phrase > 
transitive verb —> passes, and this sentence does not contain 
passes. 

5. a) S > 1A=>10B= 101A => 10108 = 10101 b) Because 
of the productions in this grammar, every 1 must be followed 
by a 0 unless it occurs at the end of the string. c) All strings 
consisting of a 0 or a 1 followed by one or more repetitions of 
01 

7 S = OSI = OOSIL = 000S111 = 000111 
9aS > OS => OOS => OO0SI > OOSI1 > 
00S111 = 00S1111=> 001111 b) S= 0S = 00S => 
001A = 0011A = 00111A = 001111 11. S = OSAB = 
OOSABAB = OOABAB = 00 AABB = OOIABB = 
0011BB = 00112B = 001122 I3.a)S > 0,5 > 1, 
S—>11 b)S> 18,534 c)S—>0Al,A > 1A,A > OA, 
A > A d)S > 0A,A > 11A,A > å 15.a)S — 005S, 
S — A b) S > 10A,A — 00A,A > A c)S > AAS, 
S > BBS, AB — BA, BA > AB,S > 4,A > 0,B > 1 
d) S — 0000000000A, A > 0A, A > A e) S > AS, S —> ABS, 
S > A, AB > BA, BA > AB,A > 0,B > 1 f)S > ABS, 
S > à, AB > BA, BA > AB,A > 0,B > 1 g)S — ABS, 
S>T,S—U,T > AT,T >A, U > BU, U > B, AB > BA, 
BA > AB,A > 0,B—1 = 17.a)S—>0S,S > A b) S > AO, 
A > 1A,A > A œ)S > 000S,S > A 19. a) Type 2, 
not type 3 b) Type 3 c) Type 0, not type 1 d) Type 2, not 
type 3 e) Type 2, not type 3 f) Type 0, not type 1 g) Type 
3 h) Type 0, not type 1 i) Type 2, not type 3 j) Type 2, 
not type 3 21. Let S, and S, be the start symbols of G; and 
G,, respectively. Let S be a new start symbol. a) Add S and 
productions S > S, and S > S,. b) Add S and production 
S > S,S,. ce) Add S and production S$ > A and S > S,S. 


23. a) sentence 


noun phrase 


JIN 


article adjective noun intransitive verb 


intransitive verb phrase 


the happy hare runs 


b) sentence c) 
noun phrase intransitive verb phrase noun phrase 
intransitive \ S/o N 
article adjective noun verb adverb article noun 
the sleepy tortoise runs quickly the tortoise 
d) sentence 
noun phrase transitive verb phrase noun phrase 
article adjective noun transitive verb article adjective noun 

the sleepy hare passes the happy tortoise 
25. a) Yes b) No c) Yes d) No 
37. signed integer 

sign integer 
— digit integer 
1 digit integer 
0 digit 
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sentence 


transitive verb phrase noun phrase 


transitive verb article noun 


passes hare 
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29. a) S > (sign) integer) 
S > (sign) (integer) . (positive integer) 
(sign) > + 
(sign) > — 
(integer) > (digit) 
(integer) > (integer) (digit) 
(digit) > i, i = 1, 2,3, 4,5, 6, 7, 8, 9,0 
(positive integer) > (integer) {nonzero digit) 
(positive integer) > (nonzero digit) (integer) 
(positive integer) > (integer)(nonzero digit) 
(integer) 
(positive integer) — (nonzero digit) 
(nonzero digit) > i, i = 1, 2, 3, 4,5, 6, 7, 8,9 
b) (signed decimal number) ::= (sign){integer) | 
(sign) (integer) . (positive integer) 
(sign) = +|- 
(integer) ::= (digit) | (integer) (digit) 
(digit) :=O0|1|[2|3|4|5]/6|7/8|9 
(nonzero digit) :=1|2|3|4|5|6|/7|8|9 
(positive integer) ::= (integer){nonzero digit) | 
(nonzero digit) (integer) | (integer) 
(nonzero integer) (integer) | (nonzero digit) 


© signed decimal number 
Par a 
sign integer uj positive integer 
e 
— integer digit nonzero digit 
digit 1 4 
3 


31. a) (identifier) ::= (Icletter) | (identifier) (Icletter) 
(Icletter) :=a|b|c|--|z 

b) (identifier) ::= (Icletter)(Icletter){Icletter) | (Icletter)(Icletter) (Icletter) {Icletter) | 
(Icletter) (Icletter){Icletter){Icletter) (Icletter) | 
(Icletter) (Icletter)(Icletter) (Icletter) (Icletter) (Icletter) 

[bl cle |z 

(ucletter) | (ucletter) (letter) | (ucletter)(letter) (letter) | 
(ucletter) (letter) (letter) (letter) | (ucletter) (letter) (letter) (letter) (letter) | 
(ucletter) (letter) (letter) (letter) (letter) (letter) 

(letter) ::= (Icletter) | (ucletter) 

(Icletter) :=a|b|c|--|z 

(ucletter) :=A|B|C|--|Z 


(Icletter) ::= a 
c) (identifier) ::= 
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d) (identifier) ::= (Icletter)(digitorus) (alphanumeric) (alphanumeric) (alphanumeric) | 
(Icletter) (digitorus) (alphanumeric) (alphanumeric) alphanumeric) (alphanumeric) 
(digitorus) ::= (digit) | _ 
(alphanumeric) ::= (letter) | (digit) 
(letter) ::= (Icletter) | (ucletter) 
(Icletter) :=a|b|c|--|z 
(ucletter) :=A|B|C|- |Z 
(digit) :=0|1|2|-19 
33. (identifier) ::= (letterorus) | (identifier) (symbol) 
(letterorus) ::= (letter) | _ 
(symbol) ::= (letterorus) | (digit) 
(letter) ::= (Icletter) | (ucletter) 
(Icletter) :=a|b|c|-+--|z 
(ucletter) :=A|B|C|-+-|Z 
(digit) ::=0|1|2|--|9 
35. numeral ::= sign? nonzerodigit digits decimal? | sign? 0 decimal? 
sign = + | — 
nonzerodigit :=1 |2|- |9 
digit ::= 0 | nonzerodigit 
decimal ::= .digit* 
37. identifier ::= letterorus symbol * 
letterorus ::= letter | _ 
symbol ::= letterorus | digit 
letter ::= Icletter | ucletter 
Icletter ::=a|b|c|--|z 
ucletter :=A|B|C|+-|Z 
digit ::=O|1|2|--|9 
39. a) (expression) 
(term) (term) (addOperator) 
(factor) (factor) (factor) (mulOperator)(addOperator) 
(identifier) (identifier) (identifier) (mulOperator)(addOperator) 
abc*+ 
b) Not generated 
c) (expression) 
(term) 
(factor) (factor) (mulOperator) 
(expression) (factor){mulOperator) 
(term) (term)(addOperator) (factor) (mulOperator) 
(factor) factor) (addOperator) (factor) (mulOperator) 
(identifier) (identifier) (addOperator) (identifier) (mulOperator) 
xy—z* 
d) (expression) 
(term) 
(factor) (factor) (mulOperator) 
(factor) {expression){mulOperator) 
(factor) (term) (mulOperator) 
(factor) (factor) (factor) (mulOperator){mulOperator) 
(factor) (factor) (expression)(mulOperator)(mulOperator) 
(factor) (factor) (term) (term) (addOperator){mulOperator)(mulOperator) 
(factor) (factor) (factor) (factor) (addOperator){mulOperator){mulOperator) 
(identifier) (identifier) (identifier) (identifier) (addOperator) (mulOperator)(mulOperator) 
wxyz—*/ 
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e) (expression) 
(term) 
(factor)(factor){mulOperator) 
(factor)( expression) (mulOperator) 
(factor) term){term)(addOperator)(mulOperator) 
(factor) (factor) (factor) (addOperator) (mulOperator) 
(identifier) (identifier) (identifier) (addOperator){mulOperator) 
ade-— x» 
41. a) Not generated 
b) (expression) 
(term)(addOperator) (term) 
(factor){mulOperator) (factor) (addOperator) (factor) (mulOperator) (factor) 
(identifier) (mulOperator) (identifier) (addOperator) (identifier) (mulOperator) (identifier) 
a/b+c/d 
c) (expression) 
(term) 
(factor)(mulOperator) (factor) 
(factor){mulOperator) ((expression)) 
(factor)(mulOperator) ((term) (addOperator){term)) 
(factor)(mulOperator) ((factor)(addOperator) (factor)) 
(identifier) (mulOperator) ((identifier) (addOperator) identifier) ) 
m * (n+p) 
d) Not generated 
e) (expression) 
(term) 
(factor) (mulOperator) (factor) 
((expression) ){mulOperator) ({expression)) 
((term)(addOperator){ term) )({mulOperator) ({term)(addOperator)(term)) 
((factor)(addOperator) (factor) ){mulOperator) (factor) (addOperator) (factory) 
({identifier) (addOperator) (identifier) )(mulOperator) ((identifier) (addOperator) (identifier) ) 
(m+n) *(p-@) 
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Section 13.2 


1.a) 


3.a)01010 b)01000 c) 11011 5.a)1100 b)00110110 c) 11111111111 


7. 25,0 25,0 25,5 25,10 25,15 25,20 


Start 


x EC, R, G} 


€ = cola 
& = root beer 
g = ginger ale 


v = Valid ID a = "Enter user ID" 

i = Invalid ID b = "Enter password" 
p = Valid password c = Prompt 

q = Invalid password x= Any input 


S-90 Answers to Odd-Numbered Exercises 


13. {5, 10, 25}, open 


10, nothing 10, nothing 


25, open 


25, open 


15. Let sọ be the start state and let sı be the state representing 
a successful call. From sọ, inputs of 2, 3, 4, 5, 6, 7, or 8 send 
the machine back to sọ with output of an error message for the 
user. From sọ an input of 0 sends the machine to state s}, with 
the output being that the 0 is sent to the network. From sọ an 
input of 9 sends the machine to state s, with no output; from 
there an input of 1 sends the machine to state s} with no out- 
put; from there an input of 1 sends the machine to state s, with 
the output being that the 911 is sent to the network. All other 
inputs while in states s or s send the machine back to sy with 
output of an error message for the user. From sọ an input of 1 
sends the machine to state są with no output; from s4 an input 
of 2 sends the machine to state s; with no output; and this path 
continues in a similar manner to the 911 path, looking next 
for 1, then 2, then any seven digits, at which point the machine 
goes to state sı with the output being that the ten-digit input 
is sent to the network. Any “incorrect” input while in states s5 
or sę (that is, anything except a 1 while in s; or a 2 while in 
S6) sends the machine back to sọ with output of an error mes- 
sage for the user. Similarly, from s4 an input of 8 followed by 
appropriate successors drives us eventually to s,, but inappro- 
priate outputs drive us back to sy with an error message. Also, 
inputs while in state s4 other than 2 or 8 send the machine back 
to state sọ with output of an error message for the user. 


17. 1,0 


10, nothing 


5, nothing 


19. R,1 
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21. 23. a) 11111 
b) 1000000 
State c) 100011001100 
So 
Sy 
So 
Section 13.3 


1. a) {000,001,1100, 1101} b) {000,0011,010,0111} 
c) {00, O11, 110, 1111} d) {000000, 000001, 000100, 
000101, 010000, 010001, 010100, 010101} 3.A={1, 101}, 
B = {0,11,000}; A = {10,111,1010, 1000, 10111, 101000}, 
B = {A};A = {4,10}, B = {10, 111, 1000} or A = {A}, 
B = {10, 111, 1010, 1000, 10111, 101000} 5. a) The set 
of strings consisting of zero or more consecutive bit pairs 10 
b) The set of strings consisting of all 1s such that the num- 
ber of 1s is divisible by 3, including the null string c) The 
set of strings in which every | is immediately preceded by 
a0 d) The set of strings that begin and end with a 1 and 
have at least two 1s between every pair of Os 7. A string 
is in A* if and only if it is a concatenation of an arbitrary 
number of strings in A. Because each string in A is also in 
B, it follows that a string in A* is also a concatenation of 
strings in B. Hence, A* C B*. 9.a) Yes b) Yes c) No 
d)No e)Yes f) Yes 11l.a) Yes b)No c) Yes d)No 
13. a) Yes b) Yes c)No d)No e)No f)No 15. We 
use structural induction on the input string y. The basis step 
considers y = A, and for the inductive step we write y = wa, 
where w € J* anda € J. For the basis step, we have 
xy = x, so we must show that f(s, x) = f(f(s, x), A). But 
part (i) of the definition of the extended transition func- 
tion says that this is true. We then assume the inductive 


hypothesis that the equation holds for w and prove that 
f(s, xwa) = f(f(s, x), wa). By part (ii) of the defini- 
tion, the left-hand side of this equation equals f(f(s, xw), a). 
By the inductive hypothesis, f(s, xw) = f(f(s, x), w), 
so f(f(s, xw), a) = fEECs, x), w), a). The right- 
hand side of our desired equality is, by part (ii) of the 
definition, also equal to f(f(f(s, x), w), a), as desired. 
17. {0, 10, 11}{0, 1}* 19. {0"1” | m > Oandn > 
1} 21. {A} U {0}{1}*{0}u {10,11}{0, 1}*U{0}{1}*{01} 
{0, 1}* U {O}{1}* {00} {O}*{1}{0, 1}* 23. Let s, be the 
only final state, and put transitions from s, to itself on either 
input. Put a transition from the start state sọ to sı on input 0, 
and a transition from sı to s, on input 1. Create state s3, and 
have the other transitions from sọ and s, (as well as both tran- 
sitions from s3) lead to s;. 25. Start state sp, only final state 
$3; transitions from sy to sọ on 0, from sọ to sı on 1, from s; 
to s, on 0, from sı to s; on 1, from s, to sy on 0, from s, to 
s3 on 1, from s; to s; on 0, from s, to son 1 27. Have five 
states, with only s, final. For i = 0, 1, 2, 3, transition from s, to 
itself on input 1 and to s,,,; on input 0. Both transitions from 
s4 are to itself. 29. Have four states, with only s, final. For 
i = 0, 1, 2, transition from s; to s,,; on input 1 but back to sọ 
on input 0. Both transitions from s, are to itself. 
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0, 1 


33. Start state sp, only final state s,; transitions from sọ to sọ 
on 1, from sọ to s; on 0, from s; to sı on 1; from s, to sy on 0 


35. 


37. Suppose that such a machine exists, with start state so 
and other state s}. Because the empty string is not in the 
language but some strings are accepted, we must have s, 
as the only final state, with at least one transition from sọ 
to sı. Because the string O is not in the language, the tran- 
sition from sọ on input O must be to itself, so the transi- 
tion from sy on input 1 must be to s,. But this contradicts 
the fact that 1 is not in the language. 39. Change each fi- 
nal state to a nonfinal state and vice versa. 41. Same ma- 
chine as in Exercise 25, but with sọ, s}, and s, as the final 
states 43. {0,01,11} 45. {4,0} u {0O"1"|m>1Ln > 1} 
47. {10" | n >0}U {10"10" | n,m > 0} 49. The union of 
the set of all strings that start with a 0 and the set of all strings 
that have no Os 


53. Add a nonfinal state s, with transitions to s; from sọ on 
input 0, from s, on input 1, and from s} on input 0 or 1. 


55. a) Start 
ee 


57. Suppose that M is a finite-state automaton that accepts 
the set of bit strings containing an equal number of Os and 
1s. Suppose M has n states. Consider the string 0"*!1"*!. By 
the pigeonhole principle, as M processes this string, it must 
encounter the same state more than once as it reads the first 
n + 1 Os; so let s be a state it hits at least twice. Then k Os in 
the input takes M from state s back to itself for some positive 
integer k. But then M ends up exactly at the same place after 
reading 0"+!+*1 "+1 as it will after reading 0”+! 1”+!, Therefore, 
because M accepts 0”+11”+! it also accepts O"++!1"+!, which 
is a contradiction. 59. We know from Exercise 58d that the 
equivalence classes of R, are a refinement of the equivalence 
classes of R,_, for each positive integer k. The equivalence 
classes are finite sets, and finite sets cannot be refined indef- 
initely (the most refined they can be is for each equivalence 
class to contain just one state). Therefore, this sequence of re- 
finements must remain unchanged from some point onward. 
It remains to show that as soon as we have R, = R,,,,, then 
R, = Rn for all m > n, from which it follows that R, = R,,, 
and so the equivalence classes for these two relations will be 
the same. By induction, it suffices to show that if R, = Rp41 
then R,,, = R45. Suppose that R,,, # R,4.. This means 
that there are states s and ż that are (n + 1)-equivalent but not 


(n+2)-equivalent. Thus, there is a string x of lengthn+2 such 
that, say, f(s, x) is final but f(t, x) is nonfinal. Write x = aw, 
where a E I. Then f(s, a) and f(t, a) are not (n+ 1)-equivalent, 
because w drives the first to a final state and the second to a 
nonfinal state. But f(s, a) and f (t, a) are n-equivalent, because 
s and t are (n + 1)-equivalent. This contradicts the fact that 
R, = R,- 61. a) By the way the machine M was con- 
structed, a string will drive M from the start state to a final 
state if and only if that string drives M from the start state to a 
final state. b) For a proof of this theorem, see a source such as 
Introduction to Automata Theory, Languages, and Computa- 
tion (2nd Edition) by John E. Hopcroft, Rajeev Motwani, and 
Jeffrey D. Ullman (Addison-Wesley, 2000). 


Section 13.4 


1. a) Any number of 1s followed by a0 b) Any number of 
Is followed by one or more Os c) 111 or 001 d)A string 
of any number of Is or 00s or some of each in a row e)A 
or a string that ends with a | and has one or more Os be- 
fore each 1 f) A string of length at least 3 that ends with 00 
3.a)No b) No c) Yes d) Yes e) Yes f) No g) No h) Yes 
5.a)0U11U 010 b) 000000" c)(0u1)X((0u 1)0 u 1)* 
d) 0*10* e) 12U01U001)* 7. a) 00*1 b) OUHNOUNHOU 
1)*0000* c)0*1* U 1*0* d) 11(111)*(00)* 9.a) Have the 
start state sọ, nonfinal, with no transitions. b) Have the start 
State So, final, with no transitions. c) Have the nonfinal start 
State sọ and a final state sı and the transition from sọ to sı 
on input a. 11. Use an inductive proof. If the regular ex- 
pression for A is Ø, A, or x, the result is trivial. Otherwise, 
suppose the regular expression for A is BC. Then A = BC 
where B is the set generated by B and C is the set generated 
by C. By the inductive hypothesis there are regular expres- 
sions B’ and C’ that generate B® and CF, respectively. Be- 


cause A? = (BC)? = C®BR, C’B’ is a regular expression for 
AX. If the regular expression for A is B U C, then the regular 
expression for A? is B’ UC’ because (BU C)? = (B®) u (CP). 
Finally, if the regular expression for A is B*, then it is easy to 
see that (B’)* is a regular expression for A®. 
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b) 


15.5 + 0A, S —> 1B, S — 0, A > OB, A > 1B,B > 
0B,B > 1B 17.S > 0C,S > 1A,S > 1,A > IA, 
A > 0C,A > 1,B — 0B,B — 1B,B > 0,B > 1, 
C > 0C, C > 1B,C — 1. 19. This follows because 
input that leads to a final state in the automaton corresponds 
uniquely to a derivation in the grammar. 21. The “only if” 
part is clear because / is finite. For the “if” part let the states 
be Sor Se Sater Bee where n = I(x). Because n > |S|, 
some state is repeated by the pigeonhole principle. Let y be 
the part of x that causes the loop, so that x = uyv and y sends 
5; tO Sj, for some j. Then uy*v € L(M) for all k. Hence, L(M) 
is infinite. 23. Suppose that L = {O"1"n = 0,1,2...} 
were regular. Let S be the set of states of a finite-state ma- 
chine recognizing this set. Letz = 071" where 3n > |S]. 
Then by the pumping lemma, z = 07"1" = www, Iv) > 1, 
and uv'w € {07"1” | n > 0}. Obviously v cannot contain 
both 0 and 1, because v? would then contain 10. So v is all Os 
or all 1s, and hence, uv?w contains too many Os or too many 
ls, so it is not in L. This contradiction shows that L is not 


regular. 25. Suppose that the set of palindromes over {0, 1} 


S 


were regular. Let S be the set of states of a finite-state machine 
recognizing this set. Let z = 0"10", where n > |S|. Apply the 
pumping lemma to get uv'w € L for all nonnegative integers i 
where /(v) > 1, and /(uv) < |S|, and z = 010" = uvw. Then v 
must be a string of Os (because n > ||), so uv?w is not a palin- 
drome. Hence, the set of palindromes is not regular. 27. Let 
z=1;then 111 ¢ Lbut 101 € L, so 11 and 10 are distinguish- 
able. For the second question, the only way for 1z to be in L 
is for z to end with 01, and that is also the only way for 11z to 
be in L, so 1 and 11 are indistinguishable. 29. This follows 
immediately from Exercise 28, because the n distinguishable 
strings must drive the machine from the start state to n differ- 
ent states. 31. Any two distinct strings of the same length 
are distinguishable with respect to the language P of all palin- 
dromes, because if x and y are distinct strings of length n, then 
xx? € P but yx? ¢ P. Because there are 2” different strings of 
length n, Exercise 29 tells us that any deterministic finite-state 
automaton for recognizing palindromes must have at least 2” 
states. Because n is arbitrary, this is impossible. 


Section 13.5 


1. a) The nonblank portion of the tape contains the string 1111 
when the machine halts. b) The nonblank portion of the tape 
contains the string 011 when the machine halts. c) The non- 
blank portion of the tape contains the string 00001 when the 
machine halts. d) The nonblank portion of the tape contains 
the string 00 when the machine halts. 3. a) The machine 
halts (and accepts) at the blank following the input, having 
changed the tape from 11 to 01. b) The machine changes ev- 
ery other occurrence of a 1, if any, starting with the first, to 
a 0, and otherwise leaves the string unchanged; it halts (and 
accepts) when it comes to the end of the string. 5. a) Halts 
with 01 on the tape, and does not accept b) The first 1 (if any) 
is changed to a 0 and the others are left alone. The input is not 
accepted. 

7. (So, 0, sp 1, R), (So, 1, 59, 1, R) 9. (59, 0, Sq, 0, R), 
(so, L sp L R), (s, s 0, R), (s 1, s, 0, R) 11. (so, O, 
51, 0, R), (Sq. l, 59, 0, R), (s 0, s 0, R), (s 1, Sọ 0, R), 
(Ssp B, Sy, B, R) 13. (sọ 0, 59, 0, R), (sœ L sp 1, R), 
(s,, 0, 5, 0, R), (si 1, 59, 1, R), (Sọ B, s2, B, R) 15. If the 
input string is blank or starts with a 1 the machine halts in 
nonfinal state sọ. Otherwise, the initial 0 is changed to an M 
and the machine skips past all the intervening Os and 1s until 
it either comes to the end of the input string or else comes to 
an M. At this point, it backs up one square and enters state sz. 
Because the acceptable strings must have a 1 at the right for 
each 0 at the left, there must be a 1 here if the string is accept- 
able. Therefore, the only transition out of s, occurs when this 
square contains a 1. If it does, the machine replaces it with 
an M and makes its way back to the left; if it does not, the 
machine halts in nonfinal state s,. On its way back, it stays in 
s3 as long as it sees 1s, then stays in s4 as long as it sees Os. 
Eventually either it encouters a 1 while in są at which point 
it halts without accepting or else it reaches the rightmost M 
that had been written over a 0 at the start of the string. If it 
is in s; when this happens, then there are no more Os in the 
string, so it had better be the case that there are no more 1s 
either; this is accomplished by the transitions (s3, M, s;, M, R) 
and (s5, M, Sẹ M, R), and s¢ is a final state. Otherwise the ma- 
chine halts in nonfinal state s;. If it is in są when this M is 
encountered, things start all over again, except now the string 
will have had its leftmost remaining 0 and its rightmost re- 
maining 1 replaced by Ms. So the machine moves, staying in 
state s4, to the leftmost remaining 0 and goes back into state 
Sq to repeat the process. 


17. (So, B, Sọ B, L), (Sp, 0, s 0, D), (si, B, E, R), 
(s M, so, M, R), (52, 0, s3, M, R), (s3, 0, 53, 0, R), 
(s3, M, s3, M, R), (s3 l, s, M, R), (Sy 1, 1, R), 
(s4, M, sy, M, R), (sy, 2, s; M, R), (Ss, 2, 55, 2, R), 
(s5, B, S6 B, L), (86, M, Sg, M, L), (S6 2, 57, 2, L),(s7, 0, s7, 0, L), 
(s7, 1, sz, 1, L), (S7, 2, S7, 2, L),(s7, M, s7, M, L), (s7, E, S2, E, R), 
(Sg, M, sg, M, L), (Sg, E, Sọ, E, L) where M and E are markers, 
with E marking the left end of the input 

19. (so, 1, sı B, R), (s 1l, s» B, R), (Sy, 1, s3, B, R), 
(s3, 1, sy, 1, R), (s1, B, sy, 1, R), ($2, B, 54, 1, R), (53, B, sy, 1, R) 
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21. (sœ 1, s B, R), (s, L sa, B, R), (s B, Sẹ B, R), 
(sy, 1, 53, B, R), (5, B, S6 B, R), (s3, 1, S4, B, R), (s3, B, 56, B, R), 
(sy, 1, S5, B, R), (sy, B, Sẹ B, R), (sẹ B, Sio 1, R), 
(s5; 1, S5, B, R), ($5, B, 57, 1, R), (57, B, Sg, 1, R), (Sg, B, So, 1, R), 
(So B, Sy, 1, R) 

23. (So, l, sœ l, R), (Sœ B, sp B, L), (s 1, sa 0, L), 
(55, 0, Sy, 0, L), (Sp, 1, 53, 0, R), (s2, B, Sẹ B, R), (s3, 0, 53, 0, R), 
(s3, 1, 53, 1, R), (83, B, S4 1, R), (Sy, B, 85, 1, L), (55, 1, 55, 1, L), 
(55,0, $2, 0, L), (56,0, 56, 1, R), (56, L, s7, 1, R), (56, B, 57, B, R) 
25. (Sq, 0, sœ 0, R), (so * Ss, B, R), (s3, *, s3 *, L), 
(s3, 0, 53, 0, L), (s3, 1, 53, 1, L), (83, B, So B, R), (s5, L, 55, B, R), 
(s5, 0, 55, B, R), (Ss, B, Sẹ B, L), (Sẹ B, Sẹ B, L), (sẹ 0, 87, L, L), 
(s7, 0, s7, 1, L), (So, Ls, 0, R), (s3; Ls), 1, R), (S1, *, So, *, R), 
(55, 0, $2, 0, R), (So, 1, s3, 0, L), (S2, B, S4 B, L), (S4 O, 84, L, L), 
(sy, *, Sg, B, L), (Sg, 0, Sg, B, L), (Sg, 1, 5g, B, L) 

27. Suppose that s,, is the only halt state for the Turing ma- 
chine in Exercise 22, where m is the largest state number, and 
suppose that we have designed that machine so that when 
the machine halts the tape head is reading the leftmost 1 of 
the answer. Renumber each state in the machine for Exer- 
cise 18 by adding m to each subscript, and take the union of 
the two sets of five-tuples. 29. a) No b) Yes c) Yes d) Yes 
31. (So, B, 81, 1, L), (So, 1, 51, 1, R), (51, B, Sọ 1, R) 


Supplementary Exercises 


1.a)S — 005111,5 —> A b)S — AABS, AB — BA, 
BA > AB,A > 0,B 31,8 > A œ©)S > ET,T > OTA, 
T > 1TB,T > A, 0A —> AO, 1A > Al, OB —> BO, 1B > B1, 
EA > E0, EB > E1,E > å 


3. S S 
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7. No, take A = {1, 10} and B = {0, 00}. 
{00, 000, 00000} and B = {00, 000}. 


d)3 e)2 f)4 


13. 


n 
w 
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En 
00 
w 
i 


17: a) kt] mk b) nkt! m” 


19. 


9, No, take A = 
ll.a)l b)l ¢)2 


23. Construct the deterministic finite automaton for A with 
states § and final states F. For A use the same automaton but 
with final states $ — F. 


25. a) Start 


27. Suppose that L = {1? | p is prime} is regular, and let S be 
the set of states in a finite-state automaton recognizing L. Let 
z = 1? where p is a prime with p > |S| (such a prime exists 
because there are infinitely many primes). By the pumping 
lemma it must be possible to write z = uvw with [(uv) < |S], 
I(v) > 1, and for all nonnegative integers i, uv'w €E L. Be- 


cause z is a string of all 1s, u = 1f, v = 1?, and w = 15, 
wherea +b +c = p,a+b < n,andb > 1. This means 
that uviw = 141216 = 1@tb+o+bli-1) — pPthl-) | Now take 


i= p+ 1. Then uviw = 1?"+), Because p(1 + b) is not prime, 
uviw ¢ L, which is a contradiction. 29. (so, *, S5, B, L), 
(Sp, 0, so 0, R), (s L sy, 0, R), (sp % s» x, R), 
(s,, L sy, LR), (Sy, 0, 55, 0, R), (52, 1, 53, 0, L), (52, B; Sa, 
B, L), (53, *, 53, *, L), (53, 0, s3 0, L), (83, 1, 53, 1, D), 
(s3, B, Sœ B, R), (s4 *, Sg, B, L), (s4 0, 54, B, L), (ss, O, 
Ss, B, L), (ss, B, sẹ B, R), (sẹ 0, 57, 1, R), (sẹ B, Sẹ B, R), 
(s7, 0, 57, 1, R), (57, L57, 1, R), (Sg, 0, Sg, 1, L), (Sg, 1, sg, LL) 


APPENDIXES 
Appendix 1 


1. Suppose that 1’ is also a multiplicative identity for the real 
numbers. Then, by definition, we have both 1 - 1’ = 1 and 
1-1'=1',sol’=1. 3. For the first part, it suffices to show 
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that [(—x) - y] + (x - y) = 0, because Theorem 2 guarantees 
that additive inverses are unique. Thus, [(—x) - y] + (x - y) = 
(—x + x) - y (by the distributive law) = 0 - y (by the inverse 
law) = y - 0 (by the commutative law) = 0 (by Theorem 5). 
The second part is almost identical. 5. It suffices to show 
that [((—x) - (—y)] + [-—@ - y)] = 0, because Theorem 2 
guarantees that additive inverses are unique: [(—x) - (—y)] + 
[-(~@- yl = [Cx - (y) + [Cn - y] (by Exercise 3) 
= (—x) - [(-y) + y] (by the distributive law) = (—x) - 0 
(by the inverse law) = 0 (by Theorem 5). 7. By defini- 
tion, —(—x) is the additive inverse of —x. But —x is the ad- 


ditive inverse of x, so x is the additive inverse of —x. There- 
fore, —(—x) = x by Theorem 2. 9. It suffices to show that 
(=x — y) + (x + y) = 0, because Theorem 2 guarantees 
that additive inverses are unique: (—x — y) + (x + y) 
[(—x) + (-y)] + (x + y) (by definition of subtraction) = 
[(-y) + (-—x)] + ( + y) (by the commutative law) = 
(—y) + [((—x) + (x + y)] (by the associative law) = (—y) + 
[(—x + x) + y] (by the associative law) = (—y) + (0 + y) 
(by the inverse law) = (—y) + y (by the identity law) = 
O (by the inverse law). 11. By definition of division and 
uniqueness of multiplicative inverses (Theorem 4) it suffices 
to prove that [(w/x) + @/2)] : Œœ- z) =w-zt+x-y. But 
this follows after several steps, using the distributive law, the 
associative and commutative laws for multiplication, and the 
definition that division is the same as multiplication by the 
inverse. 13. We must show that if x > 0 and y > 0, then 
x+y > 0. By the multiplicative compatibility law, the com- 
mutative law, and Theorem 5, we have x- y > O-y = 0. 
15. First note that if z < 0, then —z > O (add —z to both sides 
of the hypothesis). Now given x > y and —z > 0, we have 
x + (=z) > y+ (—z) by the multiplicative compatibility law. 
But by Exercise 3 this is equivalent to —(x - z) > —(y- z). 
Then add x - z and y - z to both sides and apply the various 
laws in the obvious ways to yield x -z < y-z. 17. The 
additive compatibility law tells us that w + y < x + y and 
(together with the commutative law) that x+ y < x+z. By the 
transitivity law, this gives the desired conclusion. 19. By 
Theorem 8, applied to 1/x in place of x, there is an inte- 
ger n (necessarily positive, because 1/x is positive) such that 
n > 1/x. By the multiplicative compatibility law, this means 
thatn-x>1. 21. We must show that if (w, x) ~ (w’, x’) and 
(y,z) ~ O’, z’), then (w +y, x +z) ~ (Ww! +y’, x’ +z’) and that 
(Wey+x-z,x-y+w-z) ~ (w -y +x z’, x -y +w- 2’). Thus, 
we are given that w +x’ = x+ w’ and that y +z’ = z + y’, and 
we want to show that w+ y +x’ +2’ =x+z+w' +y’ and that 
weytx-ztx’-y+w'!-2) =x- y+w-z+w -y +x z. For the 
first of the desired conclusions, add the two given equations. 
For the second, rewrite the given equations as w — x = w' — x' 
and y — z = y’ — z’, multiply them, and do the algebra. 
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Appendix 2 


1.a) 23? b) 2° ©) 2? 


5. 


3.a)2y b) 2y/3 ©) y/2 


Appendix 3 


1. After the first block is executed, a has been assigned the 
original value of b and b has been assigned the original value 
of c, whereas after the second block is executed, b is assigned 
the original value of c and a the original value of c as well. 
3. The following construction does the same thing. 


i := initial value 

while i < final value 
statement 
i:=i+1 
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Bell numbers, 650 
Bernoulli, James, 484 
Bernoulli family tree, 781 
Bernoulli’s inequality, 351 
Bernoulli trial, 484, 520 
Bézout, Etienne, 285 
Bézout’s coefficients, 285, 324 
Bézout’s identity, 285 
Bézout’s theorem, 285-286, 324 
Bhaskaracharya, 143 
Biconditional statement, 10, 115 
logical equivalences involving, 29 
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Biconnected graphs, 719 
Bidirectional bubble sort, 246 
Bienaymé, Irenée-Jules, 515 
Bienaymé’s formula, 515, 521 
Big-O estimates, 221-224 
for combinations of functions, 
225-226 
for exponential functions, 224 
for functions, 221—223 
for logarithms, 224 
for polynomials, 221 
Big-Omega (big-Q) notation, 
226-228, 244 
Big-O notation, 217-221 
definition of, 218—221, 244 
history of, 217 
Big-Theta (big-O©) notation, 
226-228, 244 
Bi-implications, 10 
Bijection, 152, 196 
Bijective function, 152 
Bijective proofs, 433 
Binary coded decimal expansion, 872 
Binary coded decimal form, 270 
Binary expansions, 260-261 
conversion between 
octal/hexadecimal expansions 
and, 263 
Binary insertion sort, 215 
Binary relation, 599, 665 
Binary representation, 263, 324 
Binary search algorithm, 205-206, 
244, 245, 554 
recursive, 384—385 
worst-case complexity, 232-233 
Binary search trees, 793-796, 841 
Binary trees, 373-374, 784, 785, 841 
extended, 373 
full, 373-374 
with prefix codes, 798—800 
Binomial coefficients, 431, 
437, 461 
extended, 566 
identities involving, 442-443 
Binomial distribution, 485 
Binomial expression, 437 
Binomial theorem, 437—440, 462 
extended, 567 
Binomial trees, 843 
Bipartite graphs, 689-691, 771 
coloring of, 690 


complete, 691 
and matchings, 692—694 
Birthday problem, 486—487 
Bit operations, 12, 115 
Bits, 12, 115 
parity check, 306 
Bit string, 12, 115 
encoding, 798 
Bitwise operations, 13, 115 
Bitwise operators, 12, 13, 115 
Block ciphers, 313-314, 324 
Block of statements, A-14—A—15 
Blocks, Sudoku, 35 
Board, 108 
Bonferroni’s inequality, 492 
Boole, George, 3, 5, 847 
Boolean algebra, 5, 37, 847—877, 879 
abstract definition of, 853 
Boolean functions. See Boolean 
functions 
identities of, 850-852 
Boolean expressions, 848—850, 879 
dual of, 852, 855, 879 
Boolean function of degree n, 848, 
850, 879 
Boolean functions, 847—853 
Boolean expressions and, 848—850 
complement of, 849 
dual of, 852 
functionally complete set of 
operators for, 857 
implicant of, 868, 879 
minimization of, 864—877, 879 
representing, 849-850, 855-857 
self-dual, 880 
threshold, 881 
values of, 849 
Boolean power, of square zero—one 
matrix, 193 
Boolean product, 847—848, 849, 879 
Boolean product of zero—one matrix, 
192-193, 196, 236 
Boolean searches, 19—20 
Boolean sum, 847-848, 879 
Boolean variable, 12, 115, 848, 
856, 879 
Boole’s inequality, 492 
Boruvka, Otakar, 838 
Bottom-up parsing, 891 
Bound 
greatest lower, 657 


least upper, 657 
lower. See Lower bound 
for number of leaves in m-ary tree, 
790-791 
upper. See Upper bound 
Bounded lattices, 669 
Bound variable, 48, 116 
Boxes, distributing objects into, 
451-454 
distinguishable, 451—452 
indistinguishable, 451, 452—454 
Boyer-Moore majority vote 
algorithm, 216 
Breadth-first search, 826—828, 842 
depth-first search vs., 827-828 
Breaking codes, 312 
Bridge, in graphs, 718 
Brute-force algorithm, 237—238, 245 
B-tree of degree k, 843 
Bubble sort, 207—208, 245 
bidirectional, 246 
worst-case complexity of, 
234-235 
Buoyancy, principle of, A-4 
Busy beaver function, 934 
Busy beaver problem, 937 
Butane, 786 
Bytes, 261 


Cactus, 843 
Caesar, Julius, 311 
Calculus, predicate, 43 
Call graphs, 678 
connected components of, 718 
The Canterbury Puzzles 
(Dudeney), 531 
Cantor, Georg, 123, 124, 183 
Cantor diagonalization argument, 
183, 196 
Cantor digits, 461 
Cantor expansion, 270 
c-approximation algorithm, 248 
Cardano, Girolamo, 469, 470 
Cardinality, 127, 179-186, 195, 196 
of multisets, 143 
Cardinal numbers, 127 
Carmichael, Robert Daniel, 299 
Carmichael number, 299, 324 
Carroll, Lewis (C. L. Dodgson), 54 
Carry, 264 
Cartesian products, 128—130 


Cases, proof by, 97, 98—101, 116 
errors with, 100-101 
Cashier’s algorithm, 210-211 
Catalan, Eugéne Charles, 534 
Catalan conjecture, 534 
Catalan numbers, 533 
Caterpillar, 844 
Cayley, Arthur, 781, 786 
Ceiling function, 157—160, 196 
properties of, 159 
Celebrity problem, 353 
Cells, adjacent, 866 
Center vertex, 793 
Chain, 651, 669 
Character cipher, 313, 324 
Characteristic equation, 541 
Characteristic function, 164 
Characteristic roots, 541, 592 
Chebyshev, Pafnuty Lvovich, 516 
Chebyshev’s inequality, 517, 521 
Check digits, 306-308 
ISBNs, 307-308 
ISSN, 310 
parity check bits, 306 
Universal Product Code, 306-307 
Checkerboards 
definition of, 108 
standard, 108 
tilings of, 108—111, 348-349 
Chessboard 
knight’s tour on, 742 
n-queens problem on, 829 
Child, of vertex, 783, 841 
Chinese postman problem, 733-734 
Chinese remainder theorem, 
293-295, 325 
Chomp, 102-103, 363 
Chomsky, linguist Noam, 885, 891 
Chromatically k-critical graphs, 769 
Chromatic number, 772 
of graph, 763 
Chung-Wu Ho, 361 
Church, Alonzo, 935 
Church-Turing thesis, 932—933 
Ciphers 
affine, 312, 324 
autokey, 327 
block, 313-314, 324 
character, 313, 324 
monoalphabetic, 313 
shift, 311, 312, 313, 324 


transposition, 313 
Vigenère, 322 
Circuits, 771 
combinational. See Combinational 
circuits 
combinatorial, 22 
in directed graph, 629, 665 
Euler, 728—734, 772 
in graphs, 714 
Hamilton, 728, 734-739 
Circular reasoning, 94, 116 
Circular relation, 667 
Circular r-permutation of n, 436 
Citation graphs, 678-679 
Class 
congruence, 255 
equivalence. See Equivalence 
classes 
NP, 239, 935-936, 938 
P, 239, 935-936, 938 
Class A addresses, 412 
Class B addresses, 412 
Class C addresses, 412 
Classical cryptography, 311-314 
Clauses, 78 
Clique, 774 
Closed form, 575 
Closed formula, 168 
Closed interval, 123 
Closed walk, 714 
Closest-pair problem, 559-561 
Closure laws 
for addition, A-1 
for multiplication, A-1 
Closures 
for integers modulo m, 257 
Kleene, 905, 938 
Closures, of relations, 628—637, 665 
reflexive, 628-629 
symmetric, 629 
transitive, 628, 630-634 
Cloud, 320 
Clustering coefficient, 774 
COBOL, 892 
Cocks, Clifford, 315, 316 
Codeword enumeration, 532—533 
Codomain 
of functions, 147, 196 
of partial function, 161 
Coefficients 
Bézout’s, 285, 324 
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binomial, 431, 437, 461 
clustering, 774 
constant. See Constant coefficients 
multinomial, 457 
Collaboration graphs, 677, 716 
Collatz problem, 112 
Collision 
in hashing, 304 
probability in hashing functions, 
487-488 
Coloring 
of bipartite graphs, 690 
edge, 769 
of graphs, 762-767, 772, 828-829 
k-tuple, 770 
of maps of regions, 762 
of simple graphs, 763 
Combinational circuits, 859—860 
depth of, 864 
examples of, 861-862 
minimization of, 864-877 
Combinations, 431—434 
of events, 473-474, 480-481 
generalized, 445—454 
generating, 459-460 
with repetition, 446-450 
Combinatorial circuit, 22 
Combinatorial game theory, 805 
Combinatorial proof, 433, 461 
Combinatorics, 405, 461, 469 
Comments, in pseudocode, 
A-12 
Common difference, 166 
Common ratio, 166 
Communication networks, 678 
Commutative group, 258 
Commutative laws 
for addition, A—1 
for Boolean algebra, 851 
for lattices, 669 
for multiplication, A-1 
for propositions, 29 
for sets, 136 
Commutative ring, 258 
Commutativity, for integers modulo 
m, 257 
Comparable elements, 651, 666 
Compatibility laws, A-2 
Compatible talks, 534 
Compatible total ordering, 
660, 666 
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Complement 

of Boolean function, 849, 879 

of elements, 670, 847 

of fuzzy sets, 146 

of sets, 135, 136, 196 
Complementary event, 473 
Complementary graph, 702 
Complementary relations, 609 
Complementation law, 136 
Complemented lattices, 670, 853 
Complement laws, 136 
Complete bipartite graphs, 691, 771 
Complete graphs, 688, 771 
Complete induction, 355 
Complete m-ary tree, 792 
Complete matching, 693, 771 
Complete m-partite graph, 774 
Completeness property, A—2 
Complexity, of algorithms, 231-241 

average-case, 233-234, 508-510 

computational. See Computational 

complexity of algorithms 

constant, 238 

Dijkstra’s, 749 

exponential, 238, 239 

factorial, 238, 239 

linear, 238 

linearithmic, 238 

logarithmic, 238 

merge sort, 558 

polynomial, 238, 239 

practical considerations, 240-241 

space, 231, 245 

time, 231-235, 244 

worst-case, 232—233, 245 
Complex numbers, sets of, 123 
Composite, 271, 324, 937 

of relations, 606—607 
Composite key, 613, 665 
Composition rule, 394 
Compositions, of functions, 

153-156, 196 

Compound interest, 169 
Compound propositions, 3, 26, 115 

consistent, 115 


equivalences, in Boolean algebra, 848 


equivalent, 9 

logically equivalent, 27, 115 
satisfiable, 33, 115 

truth tables of, 11 
unsatisfiable, 33 


Computable functions, 185, 196, 
934, 938 
Computational complexity, 231, 
933-936 
Computational complexity of 
algorithms 
of depth-first search 
algorithms, 826 
Diykstra’s algorithm, 749 
Computational geometry, strong 
induction in, 359-361 
Computation models, 885-936 
finite-state machines, 885, 
897-902, 904-914 
language. See Language 
pushdown automaton, 925 
Turing machine, 885, 924, 925 
Computer arithmetic with large 
integers, 295-297 
Computer file systems, 787 
Computer networks, 673-674 
local area networks, 695 
multicasting over, 823 
Computer representation, of sets, 
141-142 
Concatenation, 371, 904, 938 
Conclusion, 54, 74, 116 
of conditional statement, 6 
Concurrent processing, 679 
Conditional constructions, 
A-12-A-13 
Conditional-disjunction 
equivalence, 28 
Conditional expectation, of random 
variables, 518 
Conditional probability, 477, 
481-482, 520 
Conditional statements, 6-10 
contrapositive of, 9 
converse of, 9 
definition of, 6 
direct proof of, 86 
inverse of, 9 
logical equivalences involving, 29 
for program correctness, 
394-396 
truth table for, 6 
Conditions, don’t care, 872—873 
Confidence, 618, 665 
Congruence class, 255 


Congruence classes modulo m, 
642, 666 
Congruence modulo m, 640-641 
Congruences, 254 
applications of, 303-308 
linear, solving, 290-300 
Congruent to b modulo m, 254 
Conjectures, 85, 116 
3x + 1, 112-113 
Catalan, 534 
Frame’s, 539 
Goldbach’s, 279 
twin primes, 279-280 
Conjunction, 4 
definition of, 115 
distributive law of disjunction 
over, 28 
rules of inference, 76 
truth table for, 4 
Conjunctive normal form, 856, 858 
Connected components, of graphs, 
717-718, 772 
Connected graphs, 714-724, 771 
directed, 721-722 
strongly, 721 
undirected, 716-718 
weakly, 721 
Connecting vertices, 685 
Connectives, 4 
Connectivity relation, 631, 665 
Consequence, of conditional 
statement, 6 
Consistent compound 
propositions, 115 
Consistent system specifications, 
18-19 
Constant coefficients 
linear homogenous recurrent 
relations with, 
541-547, 592 
linear nonhomogenous recurrent 
relations with, 
547-550, 592 
Constant complexity, 238 
Constants, 540 
Constructions 
conditional, A-12—A—13 
loop, A-14—A-15 
Constructive existence proofs, 
101, 116 
Context-free grammars, 889, 938 


Context-free language, 889 
Context-sensitive grammars, 889 
Context-sensitive language, 889 
Contingency, 26, 115 
Continuum hypothesis, 185-186, 196 
Contradiction, 26, 27, 115 
proofs by, 90-93, 116 
Contraposition, proof by, 87—90, 116 
Contrapositive, of conditional 
statements, 9, 115 
Converse 
of conditional statements, 9, 115 
of directed graphs, 702 
Conversion 
base, 261—263 
between binary and hexadecimal 
expansions, 263 
between binary and octal 
expansions, 263 
Convex polygon, 359 
Cook, Stephen, 240 
Cook-Levin theorem, 240 
Corollary, 85, 116 
Correctness 
of algorithm, 203 
program. See Program correctness 
of recursive algorithms, 385-386 
Correspondence, one-to-one, 152, 
154, 196 
Countable sets, 180-183, 196 
Counterexamples, 44, 93, 107, 116 
Countess of Lovelace (Augusta Ada), 
31.32 
Counting, 405—462 
advanced techniques. See 
Advanced counting techniques 
bit strings, 408 
by distributing objects into boxes, 
451-454 
functions, 407 
generating functions and, 569-572 
Internet addresses, 412 
one-to-one functions, 407 
passwords, 411 
paths between vertices, 723-724 
permutations, 428—431 
pigeonhole principle and, 
420-426 
principles, 406-411 
problems, 405, 411-412 
subsets of finite sets, 344—346, 408 


Count of an itemset, 617, 665 
A Course of Pure Mathematics, 102 
Covariance, of random variables, 520 
Covering relation, 655 
Covers, 655, 770, 875 
C programming language, 642—643 
Crawl frontier, 832 
Cross edge, 834 
Crossing number, 761 
Cryptanalysis, 312-313, 324 
Cryptographic protocols, 319-320 
Cryptography, 310-321 

classical, 311-314 

public key, 310, 314-315 
Cryptosystems, 314, 324 

fully homomorphic, 321, 324 

Paillier, 323 

private key, 314-315 

public key, 315 

RSA system, 315-316, 324 
Curse of dimensionality, 535 
Cut edge, 718, 771 
Cut set, 843 
Cut vertices, 718, 771 
Cycle, 771 

in directed graph, 629, 665, 688 
Czekanowski, Jan, 838 


Databases 

attribute of, 612 

composite key, 613 

fields, 612 

intension, 612 

primary key, 612-613 

records, 612 

and relations, 612-613 
Data compression, 799 
Datagrams, 420 
Datalogy, 892 
Data mining, association rules from, 

617-619 

Datatype, 123 
Decidable problem, 933-934 
Decimal expansions, 260 
Decision problem, 933, 938 
Decision trees, 796-798, 841 
Decreasing functions, 151 
Decryption, 311 

RSA system, 317-318 
Deductive reasoning, 332 
de Fermat, Pierre, 298 
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Deferred acceptance algorithm, 216 
Deficiency, 701 
Definiteness, of algorithm, 203 
Degree 
of linear homogenous recurrence 
relations, 540 
n-ary relations, 611 
of region, 757 
of vertex in niche overlap 
graph, 686 
of vertex in undirected graph, 
685, 771 
Degree-constrained spanning 
tree, 844 
Degree of membership, of universal 
set, 146 
De La Valleé-Poussin, Charles-Jean- 
Gustave-Nicholas, 277 
Delay machine, 900, 901 
de Méré, Chevalier, 478 
De Morgan, Augustus, 27, 31, 765 
De Morgan’s laws, 27 
for Boolean algebra, 851, 855 
for propositions, 29, 30 
proving by mathematical 
induction, 344-346 
for quantifiers, 50-51 
for sets, 136 
Dense 
graph, 705, 711 
partially ordered set, 664 
Density, of undirected graphs, 711 
Denying the hypothesis, fallacies 
of, 79 
Depth-first search, 823-826, 841 
breadth-first search vs., 827-828 
in directed graphs, 830-832 
Depth of combinational circuits, 864 
Derangements, 588-591, 592 
Derivable from, 887 
Derivation, 887 
Derivation trees, 890-892, 938 
Descartes, René, 129 
Descendants, of vertex, 783, 841 
Deterministic finite-state automata, 
910, 938 
Deviation, 503 
standard, 513, 520 
Devil’s pair, 714 
Diagonal, of simple polygon, 359 
Diagonal matrix, 194 
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Diagonal relation, 628 
Diagrams 
Hasse, 654—656, 666 
state, 899 
Venn, 124-125 
Diameter, of graphs, 777 
Dictionary ordering, 458 
Die, 514-515 
dodecahedral, 522 
octahedral, 522 
Difference, A-6 
Difference equation, 539 
Differences 
backward, 539 
common, 166 
first, 539 
first forward, 595 
forward, 595 
of multisets, 143 
of sets, 135, 136, 196 
(k + 1)st, 539, 595 
symmetric, 145, 196 
Diffie-Hellman key agreement 
protocol, 319 
Digital circuits, 22-23 
Digital signatures, 320, 324 
Digraphs/directed graphs, 665, 
676, 771 
circuit/cycle in, 629, 665, 688 
connectedness in, 721—722 
converse of, 702 
dense, 705 
depth-first search in, 830-832 
edges of, 625, 687—688 
Euler circuit of, 729 
loops in, 625, 665 
paths in, 629-630, 665 
representing relations using, 
624—626 
self-converse, 775 
simple, 675 
strong components of, 
721-722, 772 
vertices, 625 
Dijkstra, Edsger Wybe, 744, 746 


Dijkstra’s algorithm, 744-749, 772 


Diophantus, 111, 298 
Dirac, Gabriel A., 736, 737 
Dirac’s theorem, 736 


Directed edges, 675, 687—688, 771 


Directed graphs. See 
Digraphs/directed graphs 

Directed multigraphs, 675, 
676, 771 

Directly derivable, 887 

Direct proof, 86-87, 116 

Dirichlet, G. Lejeune, 421 

Dirichlet drawer principle, 421 

Discrete logarithm, 300, 324 

Discrete logarithm problem, 300 

Discrete probability, 469-517 

assigning, 478—480 


of collision in hashing functions, 


487-488 


combinations of events, 473—474, 


480-481 
conditional, 477, 481-482 
finite, 470-473 
Disjoint sets, 134 
Disjunction, 4-5 
associative law for, 30 
definition of, 4, 115 
distributive law over 
conjunction, 28 
truth table for, 4 
Disjunctive normal form, 856 
Disquisitiones Arithmeticae, 255 
Distance, 834 
Distinguishable boxes 
distinguishable objects in, 
451-452 


indistinguishable objects in, 452 


Distribution 
binomial, 485 
geometric, 510-511, 520 
probability, 478, 520 
of random variable, 486, 520 
uniform, 479, 520 
Distributive lattice, 669, 853 
Distributive laws, 851-852 
for Boolean algebra, 851 
for propositions, 29 
for real numbers, A—2 
for sets, 136 


Distributivity, for integers modulo 


m, 258 
div function, 253 
Divide-and-conquer algorithms, 
553, 592 
Divide-and-conquer recurrence 
relations, 554-561 


Dividend, 253 
Divisibility, results, proving by 
mathematical induction, 
342-344 
Divisibility relation, 650 
Division 
of integers, 251, 252-254, 324 
trial, 272-273 
Division algorithm, 253-254, 324 
Division rule, 414—415, 462 
Divisor, 252, 253 
greatest common, 280-282 
DNA (deoxyribonucleic acid), 409 
Dodecahedral die, 522 
Dodgson, Charles Lutwidge (Lewis 
Carroll), 54 
Domain of definition, of partial 
function, 161 
Domains, 44, 46 
of discourse, 44, 115 
finite, 47-56 
of function, 147, 196 
n -ary relations, 611 
of partial function, 161 
restricted, quantifiers with, 48 
Dominating set, of vertices, 775 
Domination laws 
for Boolean algebra, 851 
for propositions, 29 
for sets, 136 
Dominos, 108, 334 
Don’t care conditions, 872-873, 879 
Double counting proofs, 433 
Double hashing, 308 
Double negation law, 29 
Dual 
of Boolean expression, 852, 
855, 879 
of partially ordered set, 662 
Dual graph, 762 
Duality principle, 852 
Dudeney, Henry, 531 
Dynamic programming, 533, 592 
example of, 534-536 


Ear, 364 

Eccentricity, of vertex, 793 
Edge chromatic number, 769 
Edge coloring, 769 

Edge contraction, 697—698 
Edge cut, 720, 772 


Edges, 673 
adding from graph, 697 
back, 825, 834 
connectivity, 720-721, 772 
cross, 834 
cut, 718, 771 
of digraphs, 625, 687—688 
directed, 675, 771 
endpoints of, 673 
forward, 834 
incident, 685 
matched, 693 
of multigraphs., 674 
multiple, 674, 771 
multiple directed, 675, 771 
of multiplicity, 675 
noncomplete, 688 
removing from graph, 697 
tree, 825 
undirected, 677, 771 
unmatched, 693 
Effectiveness, of algorithm, 203 
Egyptian fraction, 401 
Elementary subdivision, 758, 772 
Elements 
comparable, 651, 666 
complement of, 670, 847 
equivalent, 639 
fixed, 520 
identity, for integers modulo m, 257 
incomparable, 651, 666 
majority, 216 
maximal, in partially ordered set, 
656-658 
maximum, finding in finite 
sequence, 203 
minimal, in partially ordered set, 
656-658 
of partially ordered set, 650 
relationships between. See 
Relations 
of sets, 122, 195 
The Elements, 283 
Ellipses (...), 122 
Empty set, 124, 195 
Empty string, 166, 196, 887 
Encryption, 311, 324 
homomorphic, 310, 320-321 
private key, 324 
public key, 324 
RSA, 316-317 


Encryption key, 304, 324 
Endpoints, of edges, 673 
Enumeration, 405, 461. See also 
Counting 
codeword, 532-533 
Equality 
of functions, 148 
of sets, 123, 195 
Equal matrices, 188 
Equation 
characteristic, 541 
difference, 539 
Equivalence 
logical. See Logical equivalences 
proof of, 92-93 
Equivalence classes, 641—645 
definition of, 641, 666 
and partitions, 643-646 
representative of, 641 
Equivalence relations, 638—646, 665 
*-equivalent, 916 
Equivalent, 665 
compound propositions, 9 
Equivalent Boolean expressions, 849 
Equivalent elements, 639 
Eratosthenes, 273 
Erdős, Paul, 668 
Erdős number, 668, 716 
Errors 
with exhaustive proof, 100-101 
with proof by cases, 100-101 
in proofs, 93-95 
single, 308 
transposition, 308 
Essential prime implicant, 868, 879 
Euclid, 283 
Euclidean algorithm, 282-285, 324, 
368-369 
Euler, Leonhard, 729, 730 
Euler circuit, 728—734, 772 
Euler paths, 728-734, 772 
Euler’s criterion, 303 
Euler’s formula, 755-758, 772 
Evaluation functions, 805 
Even integer, 86 
Events, 470, 520 
combinations of, 473—474, 
480-481 
complementary, 473 
independent, 477, 482-483, 520 
probability of, 470, 520 
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Exams, scheduling, 766-767 
Exclusion rule, 370 
Exclusive or 
definition of, 5, 115 
truth table for, 6 
Exhaustive proofs, 97—98, 116 
errors with, 100-101 
Existence proofs 
constructive, 101, 116 
definition of, 101 
nonconstructive, 101—103, 116 
Existential generalization, 80 
Existential instantiation, 80 
Existential quantification, 43, 
45, 116 
Existential quantifier, 45-46 
Expansions 
balanced ternary, 269 
base-b, 260, 263 
binary, 260-261 
binary coded decimal, 872 
Cantor, 270 
decimal, 260 
hexadecimal, 261 
octal, 261 
sum-of-products, 855-856, 879 
Expected values, 503-506, 520 
in hatcheck problem, 507 
of inversions in permutation, 
507-508 
linearity of, 506-508, 521 
Experiment, 470 
Exponential complexity, 
238, 239 
Exponential functions, A-7 
big-O estimates for, 224 
Exponential generating function, 577 
Expressions 
Boolean, 848—850, 879 
infix form of, 816 
logically equivalent, 116 
prefix form of, 817 
regular, 917, 938 
Extended Backus—Naur form 
(EBNF), 896 
Extended binary trees, 373 
Extended binomial coefficients, 566 
Extended binomial theorem, 567 
Extended Euclidean algorithm, 286 
Extension, of relations, 612 
Exterior, of simple polygon, 359 
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Factor, 252 
Factorial complexity, 238, 239 
Factorial function, 160 
Facts, 85 
Failure, Bernoulli trial, 484 
Fallacies, 73, 79, 116 
of affirming the conclusion, 79 
of denying the hypothesis, 79 
Family trees, 781 
Fefferman, Charles, 278 
Female pessimal, 247 
Fermat’s last theorem, 111—112, 219 
Fermat’s little theorem, 297—298, 325 
Fibonacci, 369 
Fibonacci numbers 
formula for, 543-544 
iterative algorithm for, 387 
rabbits and, 528-529 
recursive algorithm for, 386 
Fibonacci sequence, 167—168 
Field axioms, A—1—A-—2 
Fields, databases, 612 
Final assertion, 393, 399 
Final exams, scheduling, 766-767 
Final state, finite-state 
automaton, 905 
Final value, A—14 
Finite domain, quantifiers, 47—56 
Finite graph, 673 
Finiteness, of algorithms, 203 
Finite probability, 470-473 
Finite sets, 127, 195 
Jaccard similarity of, 146 
number of subsets of, 
344-346, 408 
union of, 579 
Finite-state automata, 905—910 
designing, 907—909 
deterministic, 909-910, 938 
equivalent, 909-910 
nondeterministic, 910-914, 938 
set not recognized by, 923—924 
Finite-state machine, 885, 
897-902, 938 
for addition, 900-901 
language recognition by, 906—910 
with no output, 904-914 
with outputs, 898—902 
types of, 902 
First difference, 539 
First forward difference, 595 


Fixed elements, 520 
Fleury’s algorithm, 732, 741 
Floor function, 157—160, 196 
properties of, 159 
Floyd’s algorithm, 752 
Forests, 792, 841 
minimum spanning, 840 
spanning, 834 
Formal language, 886 
Formal power series, 564 
Formulas 
Bienaymé’s, 515, 521 
closed, 168 
Euler’s, 755-758, 772 
for Fibonacci numbers, 543—544 
Stirling’s, 160 
summations, 176, 336-340 
well-formed, 371—373, 821 
FORTRAN, 892 
Forward differences, 595 
Forward edge, 834 
Forward reasoning, 105 
Forward substitution, 169 
Four color theorem, 763—766, 772 
Frame’s conjecture, 539 
Free variable, 48, 116 
Frend, Sophia, 31 
Frequency assignments, 767 
Frequent, 617 
Frequent itemset, 617, 665 
Friendship graph, 677 
Frisbee, rocket-powered, 848 
Full adder, 862, 863, 879 
Full binary trees, 373-374 
Full m-ary trees, 784, 841 
counting vertices in, 788-789 
Full subtractor, 864 
Fully homomorphic cryptosystem, 
321, 324 
Functional completeness, 857, 879 
Functionally complete set of 
operators, for Boolean 
functions, 857, 879 
Functions, 121, 147—161, 196 
Ackermann’s, 379 
asymptotic, 231 
big-O estimates for, 221-223 
bijective, 152, 196 
Boolean. See Boolean functions 
busy beaver, 934 
ceiling, 157—160, 196 


characteristic, 164 

codomain of, 147, 196 

compositions of, 153—156, 196 

computable, 185, 196, 938 

counting, 407 

decreasing, 151 

definition of, 147 

div, 253, 267 

domain of, 147, 196 

equality of, 148 

Euler ®-function, 289 

evaluation, 805 

exponential, A-7 

factorial, 160 

floor, 157—160, 196 

generating. See Generating 
functions 

graphs of, 156-157 

greatest integer, 157 

growth of, 216-228 

hashing, 304-305 

increasing, 151 

integer-valued, 149 

inverse, 153—156, 196 

invertible, 154 

iterated, 381 

linear probing, 305 

logarithmic, A-7—A—9 

McCarthy 91, 401 

mod, 253, 267, 324 

not invertible, 154 

number-theoretic, 931 

one-to-one, 150-153, 196, 407 

one-to-one correspondence, 
152, 196 

onto, 150-153, 196, 587-588 

output, 898 

partial, 161, 196 

propositional, 41, 42, 115 

range of, 147, 196 

real-valued, 149 

recursive/inductive definitions, 
366-370, 399 

as relations, 600-601 

strictly decreasing, 151 

strictly increasing, 151 

surjective, 151 

total, 161 

transition, 898, 905, 906 

uncomputable, 185, 196, 934, 938 


Fundamental theorem of arithmetic, 
272, 325.357 
Fuzzy sets, 146 
complement of, 146 
intersection of, 146 
union of, 146 


Gale-Shapley algorithm, 216 
Gambling, 469 
Game trees, 800-805, 841 
Gates, logic, 22, 115, 858-863 
AND, 22, 859, 879 
NOT, 22 
OR, 22, 859, 879 
threshold, 881 
XOR, 881 
Gating networks, 859 
examples of, 861-862 
Gauss, Karl Friedrich, 255 
Gene, 409 
Generality, of algorithm, 204 
Generalization 
existential, 80 
universal, 79—80 
Generalized combinations, 445—454 
Generalized induction, 377—378 
Generalized permutations, 445—454 
Generalized pigeonhole principle, 
422-424, 462 
Generating functions, 563-575, 592 
and counting problems, 569-572 
exponential, 577 
ordinary, 564 
probability, 578 
proving identities via, 574-575 
for recurrence relations, 572—574 
Gene sequencing, 409 
Genome, 409 
Gentry, Craig B., 321 
Geometric distribution, 
510-511, 520 
Geometric mean, 105, 353 
Geometric progression, 166 
definition of, 196, 398 
sums of, 339-340 
sums of terms, 174 
Geometric series, 174 
Giant strongly connected component 
(GSCC), 722 
Givens, Sudoku, 35 
Goldbach, Christian, 279 


Goldbach’s conjecture, 279 
Golomb’s self-generating 
sequence, 403 
Gossip problem, 353 
Graceful tree, 844 
Grammars, 885, 886. See also 
Language 
ambiguous, 940 
Backus—Naur form, 892-894 
context-free, 889, 938 
context-sensitive, 889 
language generated by, 888 
monotonic, 890 
noncontracting, 890 
phrase-structure, 886-890 
productions of, 887 
regular, 889, 921—923, 938 
type 0, 889, 890, 938 
type 1, 889, 890, 938 
type 2, 889, 890, 938 
type 3, 889, 890, 938 
use of, 886 
Grand Hotel, Hilbert’s, 181—182 
Graphic sequence, 701 
Graphs, 673-772 
academic collaboration, 677 
acquaintanceship, 677, 715 
airline routes, 679 
bandwidth of, 776 
biconnected, 719 
bipartite, 689-694, 771 
call, 678 
chromatically k-critical, 769 
chromatic number of, 763 
circuits in, 714 
citation, 678-679 
collaboration, 677, 716 
coloring of, 762-767, 772, 
828-829 
complementary, 702 
complete, 688 
complete m-partite, 774 
connected component of, 717-718 
connectivity. See Connected graphs 
cut set of, 843 
definition of, 673 
degree sequence of, 701 
with diagnostic lines, 674 
diameter of, 777 
directed. See Digraphs 
dual, 762 
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edges. See Edges 

Euler circuit in, 728—734 

Euler paths in, 728-734 

finite, 673 

friendship, 677 

Hollywood, 677 

homeomorphic, 758-759 

infinite, 673 

influence, 677 

intersection, 683 

isomorphic, 703, 706-710 

k-connected, 719, 772 

k-tuple coloring, 770 

mixed, 675, 676 

models, 676—682 

module dependency, 679 

monotone decreasing property 
of, 777 

monotone increasing property 
of, 777 

multigraphs. See Multigraphs 

with multiple lines, 674 

with multiple one-way lines, 675 

neighbors in, 685 

niche overlap, 680, 681 

nonisomorphic, 706 

nonplanar, 758 

nonseparable, 718, 771 

paths in, 714-716 

Petersen, 741 

planar, 753-760, 772 

precedence, 679 

protein interaction, 680, 681 

pseudographs, 675, 676, 771 

radius of, 777 

regular, 702, 771 

representation of, 703-711 

simple. See Simple graphs 

simple directed, 675, 676, 771 

spanning forest of, 834 

sparse, 705, 711 

strongly connected, 721 

subgraph of, 697, 771 

terminology, 676, 685—688 

underlying undirected, 688 

undirected. See Undirected graphs 

union of, 698—699, 771 

vertices. See Vertex (vertices) 

weakly connected, 721 

web, 678, 722 

weighted. See Weighted graphs 
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Graphs of functions, 156-157 
Gray, Frank, 739 
Gray codes, 737-739 
“Greater than or equal” relation, 650 
Greatest common divisor (gcd), 
280-282, 324 
as linear combinations, 285-288 
recursive algorithm for 
computing, 383 
Greatest element, of partially ordered 
set, 656, 666 
Greatest integer function, 157 
Greatest lower bound, 657, 666 
Great Internet Mersenne Prime Links 
Search (GIMPS), 276 
Greedy algorithms, 210-212, 245, 
346-347, 800 
Growth of functions, 216—228 
big-O notation, 217—221 
Guarding set, 770 
Guthrie, Francis, 765 


Hadamard, Jacques, 277 
Haken, Wolfgang, 763 
Half adder, 862, 863, 879 
Half subtractor, 864 
Hall, Philip, 693 
Hall’s marriage theorem, 
693-694, 772 
Halting problem, 212-213, 934 
Hamilton, William Rowan, 734, 
735, 765 
Hamilton circuits, 728, 734—739, 772 
Hamilton paths, 734-739, 772 
Handle vertex, 843 
Handshaking theorem, 686—687, 772 
Hardy, Godfrey Harold, 102 
Hardy—Weinberg law, 102 
Harmonic mean, 114 
Harmonic number, 230 
inequality of, 341-342 
Harmonic series, 342 
Hashing/hashing function, 
304-305 
collision in, 304 
double, 308 
probability of collision in, 
487-488 
Hasse, Helmut, 655 
Hasse diagrams, 654—656, 666 
Hasse’s algorithm, 112 


Hatcheck problem, 507, 588-591 
Header length field, 420 
Heawood, Percy, 763 
Height, of rooted trees, 789, 841 
Hexadecimal expansions, 261 
conversion between binary 
expansion and, 263 
Hexadecimal representation, 
263, 324 
Hexagon identity, 444 
Hilbert, David, 181, 186 
Hilbert’s Grand Hotel, 181—182 
Hilbert’s Tenth Problem, 934 
Hoare, C. Anthony R., 395 
Hoare triple, 393 
Hockeystick identity, 444 
Hollywood graphs, 677 
Homeomorphic graphs, 
758-759, 772 
Homomorphic encryption, 310, 
320-321 
Hopper, Grace Brewster Murray, 911 
Hops, 696 
Horner’s method, 242—243 
Host number (hostid), 412 
Houses-and-utilities problem, 754 
Huffman, David A., 799 
Huffman coding, 799-800, 841 
Hydrocarbons, trees modeling, 786 
Hypothesis 
of conditional statement, 6 
continuum, 185-186, 196 
inductive, 333, 336, 348, 
349-350, 694 
Hypothetical syllogism, 76 


Icosian puzzle, 734 
Idempotent laws 
for Boolean algebra, 851, 855 
for lattices, 669 
for propositions, 29 
for sets, 136 
Identification numbers 
single error in, 308 
Identifiers, in C Programming 
Language, 642-643 
Identity 
Bézout’s, 285 
elements, for integers modulo 
m, 257 
elements axiom, A-1 


hexagon, 444 
hockeystick, 444 
involving binomial coefficients, 
442-443 
Pascal’s, 440—442, 462 
proving, generating functions for, 
574-575 
Vandermonde’s, 442-443 
Identity laws 
additive, A—1 
for Boolean algebra, 851 
multiplicative, A-1 
for propositions, 29 
for sets, 136 
Identity matrix, 190, 196 
if then statement, 42, 394—396, 
A-12-A-13 
Image 
of element, 147, 196 
inverse, 163 
of subsets, 149 
Implicant, 879 
essential prime, 868, 879 
prime, 868, 879 
Implication, 6, 115 
Incidence matrices, 705—706, 771 
Incident edge, 685, 771 
Inclusion—exclusion principle. See 
Principle of inclusion-exclusion 
Inclusive or, 5 
Incomparable elements, 651, 666 
Incomplete induction, 355 
Increasing functions, 151 
Increment, for linear congruential 
method, 305 
In-degree of vertex, 687, 771 
Independence number, 777 
Independent events, 477, 
482-483, 520 
Independent random variables, 
511-513, 520 
Independent vertices, 777 
Index of summation, 172-173 
Index registers, 767 
Indicator random variable, 518 
Indirect proofs, 87 
Indistinguishable boxes 
distinguishable objects in, 
452-454 
indistinguishable objects 
and, 454 


Induction 
complete, 355 
generalized, 377-378 
incomplete, 355 
mathematical. See Mathematical 
induction 
strong induction. See Strong 
induction 
structural, 366, 374-377, 399 
Inductive definitions, 366 
of functions, 366-370 
of sets, 370-374, 399 
of strings, 370 
of structures, 370-374 
Inductive hypothesis, 333, 336, 348, 
349-350, 694 
Inductive loading, 353 
Inductive reasoning, 332 
Inductive step, 333, 399 
Inequality 
Bernoulli’s, 351 
Bonferroni’s, 492 
Boole’s, 492 
Chebyshev’s, 517, 521 
harmonic numbers, 341—342 
Markov’s, 519 
proving by mathematical 
induction, 340-342 
triangle, 113, 146 
Infinite graph, 673 
Infinite ladder, 332 
strong induction and, 355-356 
Infinite series, 176-177 
Infinite set, 121, 127, 195 
Infinitude of primes, 274-276 
Infix form, 816 
Infix notation, 816-819, 841 
Influence graph, 677 
Informal proofs, 85 
Information flow, lattice model 
of, 659 
Information networks, 678-679 
Information retrieval (IR), 680-68 1 
Initial assertion, 393, 399 
Initial conditions, 167, 592 
Initial state, 898, 905 
Initial term, 166 
Initial value, A—14 
Initial vertex, 625 
Injection, 150 


Inorder traversal, 809, 811—812, 
815-816, 841 
Input, algorithms, 203 
Input alphabet, 898, 905 
Insertion sort, 208, 245 
average-case complexity of, 
509-510 
binary, 215 
worst-case complexity of, 234—235 
Instantiation 
existential, 80 
universal, 79, 80 
Integers, 251, 260 
binary coded decimal form, 270 
composite, 271 
division of, 251, 252-254 
even, 86 
fast multiplication of, 555 
large, computer arithmetic with, 
295-297 
least common multiple, 282 
nondecreasing, 214 
odd, 86 
one’s complement 
representations, 270 
pairwise relatively prime, 281, 324 
positive, 122, 251, 288, 331 
primes, 271-272 
relatively prime, 281, 324 
representations of, 260-263 
sequences, 170—172 
sets of, 122 
signed, 893 
squarefree, 591 
two’s complement 
representations, 270 
unary representations, 931 
Integer sequences, 170-172 
Integers modulo m, 257-258 
additive inverses, 257 
associativity, 257 
closure, 257 
commutativity, 257 
distributivity, 258 
identity elements, 257 
Integer-valued functions, 149 
Intension, of database, 612 
Interest, compound, 169 
Interior, of simple polygon, 359 
Interior diagonal, of simple 
polygon, 359 
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Interior vertices, 634 
Internal vertices, 783, 841 
International Standard Book Number 
(ISBN-10), 307-308 
International Standard Serial 
Number (ISSN), 310 
Internet, search engines on, 831-832 
Internet addresses, counting, 412 
Internet Protocol (IP) 
multicasting, 823 
Intersection 
of fuzzy sets, 146 
of multisets, 143 
of sets, 134, 139-141, 196 
Intersection graph, 683 
Intervals, 123 
closed, 123 
open, 123 
Intractable problem, 239, 245 
Intractable problems, 936 
Invariant for graph isomorphism, 
707, 771 
Invariants, loop, 396-398, 399 
Inverse, 291, 324 
of conditional statements, 9, 115 
multiplicative, 64 
Inverse functions, 153—156, 196 
Inverse laws 
for addition, A—2 
for multiplication, A-2 
Inverse relations, 609 
Inversions, in permutation, expected 
number of, 507-508 
Inverter, 22, 859, 879 
Invertible function, 154 
IP multicasting, 823 
IR. See Information retrieval (IR) 
Irrational numbers, 89 
Irreflexive relations, 609 
Isobutane, 786 
Isolated vertex, 686, 771 
Isomorphic graphs, 703, 
706-710, 771 
paths and, 722-723 
Items, 617 
Itemset, 617, 665 
Iterated function, 381 
Iterated logarithm, 381 
Iteration, 165, 169, 386-387, 399 
Iterative, 386 
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Iterative algorithm, for Fibonacci 
numbers, 387 


Jaccard distance, 146 
Jaccard similarity, of finite sets, 146 
James, William, 42 
Jealous husbands problem, 728 
Jigsaw puzzle, 363 
Jobs 

lateness of, 247 

slackness of, 247 
Join, 665 

in lattice, 669 

n-ary relations, 615 

of zero—one matrices, 192, 196 
Josephus problem, 538 


Kakutani’s problem, 112 
Karnaugh, Maurice, 866 
Karnaugh maps, 865, 866-872, 879 
k-connected graphs, 719, 772 
Kempe, Alfred, 763 
Kempe, Alfred Bray, 763 
k-equivalent, 916 
Key, 304, 324 

composite, 613, 665 

primary, 612-613, 665 
Key exchange, 319, 324 
Key exchange protocol, 323, 324 
Keyspace, 314 
Kissing problem, 171 
Kitab al-jabr w’al muquabala, 202 
k-itemset, 617 
(k + 1)st difference, 539, 595 
Kleene, Stephen, 905, 917 
Kleene closure, 905, 938 
Kleene’s theorem, 919—921, 939 
K-map, 865, 866-872, 879 
Knapsack problem, 248, 594 
Knight, 742 
Knight’s tour, 742 
Knuth, Donald E., 206, 217, 

220, 227 
Kronecker delta, 190 
Kruskal, Joseph Bernard, 
837, 838 

Kruskal’s algorithm, 837-839, 842 
k-tuple coloring, 770 
Kuratowski, Kazimierz, 759 
Kuratowski’s theorem, 758—760, 772 
k-vertex-connected, 719 


Labeled tree, 793 
Lady Byron, 32 
Lagarias, Jeffrey, 112 
Lamè, Gabriel, 370 
Landau, Edmund, 217 
Landau symbol, 217 
Language, 885-894, 938. See also 
Grammars 
context-free, 889 
context-sensitive, 889 
formal, 886 
generated by grammar, 888 
natural, 885 
recognition by finite-state 
machines, 906-910, 938 
recognized/accepted, 906, 912 
regular, 889 
syntax, 885 
Language over, 887 
Laplace, Pierre Simon, 469, 471, 477 
Laplace’s definition, of probability, 
470, 520 
Large integers, computer arithmetic 
with, 295-297 
Lateness, of job, 247 
Lattice point, 401 
Lattices, 658-659, 666 
associative laws for, 669 
bounded, 669 
commutative laws for, 669 
complemented, 670 
distributive, 669 
idempotent laws for, 669 
join in, 669 
lower bound of, 669 
meet in, 669 
upper bound of, 669 
Law of detachment, 75 
Law of double complement, in 
Boolean algebra, 851, 855 
Law of large numbers, 484 
Law of total expectation, 518 
Laws 
absorption. See Absorption laws 
associative. See Associative laws 
closure, 943 
commutative. See Commutative 
laws 
complement, 136 
complementation, 136 


De Morgan’s. See De Morgan’s 
laws 
distributive. See Distributive laws 
domination. See Domination laws 
of double complement, in Boolean 
algebra, 851, 855 
double negation, 29 
Hardy—Weinberg, 102 
idempotent. See Idempotent laws 
identity. See Identity laws 
inverse, A—2 
of large numbers, 484 
negation, 29 
of total expectation, 518 
transitivity, A-2 
trichotomy, A—2 
used to prove basic facts, 
A-3-A-5 
The Laws of Thought (Boole), 3, 847 
Leaf, 783, 841 
Least common multiple, 282, 324 
Least element, of partially ordered 
set, 656, 666 
Least upper bound, 657, 666, A—2 
Left child, of vertex, 785 
Left subtree, 785 
Legendre symbol, 303 
Lemma, 85, 116 
Length of path, 714 
in directed graphs, 629 
in weighted graph, 743 
Length of string, 166, 371 
Le probléme de rencontres, 589 
Level, of vertex, 789, 841 
Level order of vertex, 843 
Lexicographic ordering, 377, 458, 
652-654, 666 
Liber Abaci, 369 
Lift, of association rules, 621 
Limits, definition of, 65 
Linear array, 696 
Linear bounded automata, 925 
Linear combinations, greatest 
common divisor as, 285—288, 
324 
Linear complexity, 238 
Linear congruences, solving, 
290-300, 324 
Chinese remainder theorem, 
293-295 
Linear congruential method, 305 


Linear homogenous recurrence 
relations, 541—547, 592 
Linearithmic complexity, 238 
Linearity of expected values, 
506-508, 521 
Linearly ordered sets, 651 
Linear nonhomogenous recurrence 
relations, 547-550 
Linear probing function, 305 
Linear search algorithm, 204, 
244, 245 
average-case complexity of, 
233-234, 508-509 
recursive, 384 
Lists, merging, 389-390 
Literal, 856, 879 
Load balancing problem, 248 
Local area networks, 695 
Logarithmic complexity, 238 
Logarithmic functions, A~-7—A-9 
Logarithms 
big-O estimates for, 224 
discrete, 300, 324 
iterated, 381 
Logical equivalences, 27—30, 852 
constructing, 31-32 
quantifiers, 49 
Logical expressions 
translating from English into, 
17-18, 51-53, 66-67 
translating mathematical 
statements into, 64-65 
Logically equivalent compound 
propositions, 27, 115 
Logically equivalent expressions, 116 
Logical operators, 3, 115 
precedence of, 11 
Logic circuits, 22-23, 115 
Logic gates. See Gates, logic 
Logic programming, 55-56 
Logic puzzles, 20-22 
Longest path problem, 753 
Loop constructions, A-14—A-15 
Loop invariants, 396-398, 399 
Loops/looping, 665, 771 
in digraphs, 625, 674 
within loops, A-15 
nested, 61 
quantification and, 47, 61-62 
Lord Byron, 32 
Lotteries, 471 


Lower bound, 666 

greatest, 657, 666 

of lattices, 669 

of partially ordered set, 657 
Lower limit, of summation, 173 
Lucas, Edouard, 529 
Lucas, François Edouard, 171 
Lucas numbers, 400, 551 
Lucas sequence, 171 
Lukasiewicz, Jan, 818 


Machine minimization, 910 
Machines 
finite-state, 885, 897-902, 
904-914, 938 
Mealy, 902, 938 
Moore, 902, 903 
Turing, 885, 924, 925, 
927-936 
unit-delay, 900 
Majority element, 216 
Makespan, 248 
Male optimal, 247 
Mappings, 147. See also Functions 
Maps, coloring of, 763 
Markov’s inequality, 519 
m-ary trees, 784, 841 
balanced, 789-790 
complete, 792 
full, 784, 788-789, 841 
number of leaves in, bound for, 
790-791 
Master theorem, 558 
Matched edges, 693 
Matchings, 771 
bipartite graphs and, 692-694 
complete, 693, 771 
maximum, 693, 771 
The Mathematical Analysis of Logic 
(Boole), 5 
Mathematical induction, 31, 
331-354 
axiom, A-5 
bad of, 336 
creative uses of, 336 
definition of, 332 
good of, 336 
proofs by. See Proofs, by 
mathematical induction 
validity of, 354-355 
as valid proof technique, 334—335 
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Mathematical statements, translated 
into logical expressions, 64—65 
A Mathematician’s Apology, 102 
Matrix-chain multiplication, 236 
Matrix/matrices, 188-193 
addition of, 189, 196 
adjacency, 704-705, 
723-724, 771 
arithmetic, 189-190 
Boolean product of, 192 
definition of, 188 
diagonal, 194 
equal, 188 
identity, 190, 196 
incidence, 705-706, 771 
multiplication, 235-236, 
555-558 
powers of, 190-191 
product of, 189-190, 196 
representing relations using, 
621-624 
sparse, 705 
square, 188, 191 
symmetric, 191, 196 
transpose of, 190-191, 196 
upper triangular, 244 
zero-one, 191-193, 196 
Maurolico, Francesco, 333 
Maximal elements, 656-658 
Maximum matching, 693, 771 
Maximum satisfiability problem, 524 
Maximum spanning tree, 840 
Maxterm, 858 
McCarthy, John, 402 
McCarthy 91 function, 401 
McCluskey, Edward, 873 
Mealy, G. H., 902 
Mealy machines, 902, 938 
Mean, 214 
arithmetic, 105, 353 
geometric, 105, 353 
harmonic, 114 
quadratic, 114 
Median, 214 
Meet 
in lattice, 669 
of zero—one matrix, 192, 196 
Members, of sets, 122, 195 
Membership tables, 138, 196 
Memoization, 536 
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Merge sort, 388-391, 399, 
554-555 
complexity of, 558 
recursive, 389 
Mersenne, Marin, 275 
Mersenne primes, 275, 276, 324 
Mesh network, 696 
Metacharacters, 896 
Methods 
middle-square, 309 
probabilistic, 490—491, 520 
Quine-McCluskey, 865, 866, 
873-877 
roster, 122, 195 
Methods of proof, 86 
by cases, 97, 98-101 
by contradiction, 90-93 
by contraposition, 87—90 
direct, 86-87 
by exhaustion, 97-98 
proofs of equivalence, 92—93 
trivial, 89 
vacuous, 88 
Methodus Differentialis, 160 
Metrics, 146 
Middle-square method, 309 
Millbanke, Annabella, 32 
Miller’s test for base b, 302 
Minimal elements, 656—658 
Minimization of combinational 
circuits, 864-877 
Minimum dominating set, 775 
Minimum spanning forest, 840 
Minimum spanning trees, 835-839 
algorithms for, 835-839 
definition of, 835, 841 
Minmax strategy, 803, 841 
Minterm, 856, 879 
Mistakes, in proofs, 93-95 
Mixed graph, 675, 676 
Mode, 214 
Modeling 
computation. See Computation 
models 
with graphs, 676—682 
with recurrence relations, 
528-533 
with trees, 785-788 
mod function, 253, 267, 324 
Modular arithmetic, 251—252, 
254-258, 324 


Modular exponentiation, 267—268 
algorithm, 268 
recursive algorithm for, 384 
Modular properties, in Boolean 
algebra, 855 
Module dependency graph, 679 
Modulus, 254 
for linear congruential 
method, 305 
Modus ponens, 75, 76 
universal, 81 
Modus tollens, 76 
universal, 81—82 
Mohammed’s scimitars, 732 
Molecules, trees modeling, 786 
Monoalphabetic ciphers, 313 
Monotone decreasing, 777 
Monotone increasing, 777 
Monotonic grammars, 890 
Monte Carlo algorithms, 
488-490, 494 
Monty Hall Three Door Puzzle, 
474-475 
Moore, E. F., 902 
Moore machine, 902, 903 
Muddy children puzzle, 21 
Multicasting, Internet Protocol, 823 
Multigraphs, 674, 676, 771 
directed, 675, 676, 771 
Euler path of, 732-733 
Multilevel security policy, 659 
Multinomial coefficient, 457 
Multinomial theorem, 457 
Multiple directed edges, 675, 771 
Multiple edges, 674, 771 
Multiple output circuit, 862 
Multiples, 252 
Multiplexer, 864 
Multiplication 
algorithm, 265-266 
associative laws for, A—1 
closure laws for, A—1 
commutative laws for, A—1 
integers, 555 
inverse laws for, A—2 
matrix, 235-236, 555-558 
matrix-chain, 236 
Multiplicative compatibility law, A-2 
Multiplicative inverse, 64 
Multiplicatively homomorphic, 321 
Multiplicity 


edges of, 675 
of multisets elements, 143 
Multiplier, for linear congruential 
method, 305 
Multisets, 143—144 
cardinality of, 143 
definition of, 143, 195 
difference of, 143 
intersection of, 143 
multiplicities of elements, 143 
sum of, 143 
union of, 143 
Mutually independent events, 483, 
484, 520, 523 
Mutually reachable vertices, 726 
Mutually relatively prime, 326 


Naive set theory, 124 
Naive string matcher, 209-210 
NAND operator, 857, 879 
n-ary predicate, 42 
n-ary relations, 611-619, 665 
degree, 611 
domains, 611 
operations on, 613—616 
Natural language, 885 
Natural language understanding 
(NLU), 680-681 
Natural numbers, 123 
sets of, 121 
Naur, Peter, 892 
NAUTY (software for isomorphism 
testing), 709 
n-cube, 689, 771 
n-dimensional hypercube, 689 
Negation, 115 
of nested quantifiers, 67—68 
of propositions, 3, 4 
of quantified expression, 49-51 
truth table for, 4 
Negation laws, 29 
Negation operator, 4 
Neighbors, in graphs, 685 
Nested loops, 61 
Nested quantifiers, 60-68 
negation of, 67—68 
statements involving, 61-62 
translating into English, 65 
translating mathematical 
statements into statements 
involving, 64—65 


Network number (netid), 412 
Networks 

communication, 678 

computer. See Computer network 

gating, 859 

information, 678—679 

mesh, 696 

semantic, 681—682 

social, 676 

tree-connected, 787-788 
Niche overlap graph, 680, 681 
Nim, 802 
NLU. See Natural language 
understanding (NLU) 
Nodes. See Vertex (vertices) 
Noncomplete edges, 688 
Nonconstructive existence proofs, 
101-103, 116 
Noncontracting grammars, 890 
Nonconvex polygon, 359 
Nondecreasing integers, 214 
Nondeterministic finite-state 
automata, 910-914, 938 
Nondeterministic Turing machine, 
931-932, 938 
Nonisomorphic graphs, 706 
Nonoverlapping, 364 
Nonplanar graphs, 758 
Nonseparable graphs, 718, 771 
Nonterminals, 887 
Nonterminal symbols, 887 
NOR operator, 857, 879 
Notation 

big-O, 217-221 

big-Omega (big-Q), 226-228, 244 

big-Theta (big-O), 226-228, 244 

infix, 816-819, 841 

Polish, 817, 841 

postfix, 816-819, 841 

prefix, 816-819, 841 

reverse Polish, 817, 841 

sequence, 165 

set builder, 122 

summation, 172 
NOT gate, 22 
Not invertible function, 154 
Noun, 886, 887 
Noun phrase, 886, 887 
NP (class of nondeterministic 

polynomial-time problems), 
239, 935-936, 938 


NP-complete problems, 239-240 
n-place predicate, 42 
n-queens problem, 33-35, 829 
n-tuples, 612 
ordered, 128 
Null quantification, 59 
Null set, 124, 195 
Null string, 887 
Numbering plan, 407—408 
Numbers 
Bacon, 716 
Bell, 650 
cardinal, 127 
Carmichael, 299, 324 
Catalan, 533 
chromatic, 763, 772 
complex, 123 
crossing, 761 
edge chromatic, 769 
Erdos, 668, 716 
Fibonacci. See Fibonacci numbers 
harmonic, 230, 341—342 
independence, 777 
large, law of, 484 
Lucas, 400, 551 
natural, 123 
pseudorandom, 305-306 
Ramsey, 426 
rational, 122 
real, 122 
routing transit, 327 
Ulam, 198 
Number-theoretic functions, 931 
Number theory, 251, 310 


Objects, 124 
distinguishable, 45 1-452 
indistinguishable, 452-454 
Obligato game, 117 
Octahedral die, 522 
Octal expansions, 261 
conversion between binary 
expansion and, 263 
Octal representation, 263, 324 
Odd integer, 86 
Odd pie fights, 347-348 
Odlyzko, Andrew, 171 
One’s complement, 270 
One-to-one correspondence, 152, 
154, 196 
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One-to-one function, 150-153, 
196, 407 
On-Line Encyclopedia of Integer 
Sequences (OEIS), 172 
Onto functions, 150-153, 196, 
587-588 
Open interval, 123, 353 
Open problems, 111-113 
about primes, 278-280 
Operands, well-formed formula of, 
372-374 
Operators 
bitwise, 12, 13, 115 
logical. See Logical operators 
negation, 4 
selection, 613, 665 
well-formed formulae of, 372-374 
Opposite parity, 86 
Optimal algorithm, 243 
Optimal connectivity, 776 
Optimal for suitors, 364 
Optimization problems, 210 
Order, of quantifiers, 62—64 
Order axioms, A-1, A-2 
Ordered n-tuples, 128 
Ordered pairs, 128 
Ordered rooted tree, 785, 841 
Ordering 
lexicographic, 377, 458, 
652-654, 666 
partial, 650-661, 666 
quasi-ordering, 669 
total, 651, 660, 666 
Ordinary generating function, 564 
Ore, Øystein, 736, 737 
Ore’s theorem, 736 
Organizational tree, 786-787 
OR gate, 22, 859, 879 
Orientation of undirected graph, 776 
OR operator, 13, 19 
Out-degree of vertex, 687, 771 
Output, algorithms, 203 
Output alphabet, 898 
Output function, 898 


P (class of polynomial-time 
problems), 239, 
935-936, 938 

Paillier cryptosystem, 323 

Pairwise independent events, 
483, 520 
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Pairwise relatively prime integers, 
281, 324 
Palindrome, 214, 418, 895 
Paradoxes, 124, 195 
Russell’s, 133, 186 
St. Petersburg, 523 
Parallel algorithms, 695 
Parallel processing, 241, 695 
tree-connected, 787—788 
Parent, of vertex, 783, 841 
Parent relations, 607-608 
Parity 
opposite, 86 
same, 86 
Parity check bits, 306 
Parse trees, 890-892, 938 
Parsing 
bottom-up, 891 
top-down, 891 
Partial correctness, 393 
Partial functions, 161, 196 
codomain of, 161 
domain of, 161 
domain of definition, 161 
undefined values of, 161 
Partially homomorphic, 321 
Partially ordered set, 650, 666 
comparable elements of, 651 
dense, 664 
dual of, 662 
elements of, 650 
greatest element of, 656 
Hasse diagram of, 654—656 
least element of, 656 
maximal elements in, 656—658 
minimal elements in, 656—658 
well-founded, 664 
Partial orderings, 650-661, 666 
Partitions 
of positive integers, 380, 454, 578 
refinement of, 649 
of sets, 643-646, 666 
Pascal, Blaise, 441, 469, 478 
Pascal’s identity, 440-442, 462 
Pascal’s triangle, 440-442, 461 
Passwords, 411 
Paths, 714-716 
in acquaintanceship graphs, 715 
in collaboration graphs, 716 
counting between vertices, 
723-724 


in directed graphs, 629-630, 665 
Euler, 728-734, 772 
Hamilton, 734-739, 772 
and isomorphism, 722-723 
length of. See Length of path 
in undirected graph, 771 
Pattern, strings, 209 
Payoff, 801 
Pecking order, 776 
Peirce, Charles Sanders, 42 
Pendant vertex, 686, 771 
Perfect power, 97 
Perfect square, 87 
Permutations, 428—431, 461 
generalized, 445-454 
generating, 457-459 
with indistinguishable objects, 
450-451 
inversions in, expected number of, 
507-508 
with repetition, 446 
Petersen, Julius Peter Christian, 742 
Petersen graph, 741 
Phrase-structure grammars, 
886-890, 938 
types of, 889-890 
Pick’s theorem, 363 
Pigeonhole principle, 90, 
420-426, 462 
applications of, 424-426 
generalized, 422-424, 462 
Planar graphs, 753—760, 772 
applications of, 755 
Euler’s formula and, 755-758 
Polish notation, 817, 841 
Polygon 
convex, 359 
definition of, 359 
nonconvex, 359 
sides, 359 
simple, 359 
triangulations, 360 
vertex, 359 
Polynomial complexity, 238, 239 
Polynomials, big-O estimates 
for, 221 
Polyominoes, 110 
Poset. See Partially ordered set 
Positive integers, 122, 251, 331 
partition of, 380, 454, 578 
prime factorization of, 288 


Positive real numbers, 123 
Postconditions, 43 
Postfix form, 817 
Postfix notation, 816-819, 841 
Postorder traversal, 809, 812-814, 
815-816, 841 
Postulates, 85 
Power generator, 309 
Powers, of matrices, 190-191 
Power series, 564-567 
formal, 564 
Power sets, 128, 196 
Precedence 
of logical operators, 11 
of quantifiers, 48 
Precedence graphs, 679 
Preconditions, 43 
Predicate calculus, 43 
Predicate logic, 40 
Predicates, 40-43, 115 
logically equivalent, 49 
n-ary, 42 
n-place, 42 
truth set of, 131 
Prefix codes, 798—800 
Prefix form, 817 
Prefix notation, 816—819, 841 
Preimage, of element, 147, 196 
Premises 
of argument, 54, 73, 74, 116 
of conditional statement, 6 
Preorder traversal, 809-810, 
809-811, 814, 
815-816, 841 
Prim, Robert Clay, 837, 838 
Primary key, 612-613, 665 
Prime implicant, 868, 879 
Prime number theorem, 276-277 
Primes, 271—272, 324 
and arithmetic progressions, 278 
conjectures about, 278-280 
distribution of, 276-277 
factorization, 273 
infinitude of, 274—276 
mutually relatively, 326 
open problems about, 278—280 
primitive root of, 300, 324 
Primitive root, of primes, 300, 324 
Prim-Jarnik algorithm, 835-837 
Prim’s algorithm, 835-837, 842 
Principia Mathematica, 37, 125 


Principle of buoyancy, A-4 
Principle of inclusion—exclusion, 
134, 413, 579-583, 592 
alternative form of, 585-586 
applications of, 585-591 
Principle of mathematical induction, 
333, 398 
Principles 
counting, 406—411 
Dirichlet drawer, 421 
duality, 852 
pigeonhole, 90, 420-426, 462 
of well-ordered induction, 652 
Private key cryptosystems, 314-315 
Private key encryption, 324 
Probabilistic algorithms, 469, 
488, 520 
Probabilistic method, 490-491, 520 
Probabilistic primality testing, 
489-490 
Probabilistic reasoning, 474—475 
Probability 
assigning, 478—480 
of collision in hashing functions, 
487-488 
of combinations of events, 
473-474, 480-481 
conditional, 477, 481-482, 520 
discrete. See Discrete probability 
of events, 470, 520 
finite, 470-473 
Laplace’s definition of, 470, 520 
Probability distribution, 478, 520 
Probability generating function, 578 
Probability theory, 469, 477—491 
Problems 
art gallery, 770 
birthday, 486—487 
busy beaver, 937 
celebrity, 353 
Chinese postman, 733-734 
closest-pair, 559-561 
Collatz, 112 
counting, 405, 411-412 
decidable, 933-934 
decision, 933, 938 
discrete logarithm, 300 
gossip, 353 
halting, 212-213, 934 
hatcheck, 507, 588-591 
Hilbert’s Tenth, 934 


houses-and-utilities, 754 
intractable, 239, 245, 936 
jealous husbands, 728 
Josephus, 538 
Kakutani’s, 112 
kissing, 171 
knapsack, 248, 594 
longest path, 753 
maximum satisfiability, 524 
NP-complete, 239-240 
n-queens, 829 
open, 111-113, 278-280 
optimization, 210 
P versus NP, 239-240 
satisfiability, 37 
searching, 204 
shortest-path, 743-751, 772 
solvable, 239, 245, 933-934, 938 
Syracuse, 112 
tractable, 239, 245, 936 
traveling salesperson, 737, 743, 
749-751, 772 
two children, 523 
Ulam’s, 112 
undecidable, 934 
unsolvable, 239, 245, 934, 938 
Procedure statements, A—11 
Product 
Boolean, 847-848, 849, 879 
of matrix, 189-190 
Productions, of grammars, 887 
Product-of-sums expansion, 858 
Product rule, 406, 462 
for sets, 462 
Program correctness, 393-398, 399 
conditional statements for, 
394—396 
loop invariants for, 396—398 
partial, 393 
rules of inference for, 394 
verification, 393-394 
Programming, logic, 55-56 
Projection, 614, 665 
Prolog, 55, 56, 78 
Prolog facts, 55 
Prolog rules, 55 
Proofs, 84-85 
bijective, 433 
by cases, 97, 98-101, 116 
combinatorial, 433, 461 
by contradiction, 90-93, 116 
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by contraposition, 87—90, 116 
definition of, 85, 116 
direct, 86—87, 116 
double counting, 433 
of equivalence, 92—93 
exhaustive, 97—98, 116 
existence. See Existence proofs 
existing, adapting, 106 
indirect, 87 
informal, 85 
by mathematical induction. See 
Proofs, by mathematical 
induction 
methods of. See Methods of proof 
mistakes in, 93—95 
trivial, 89, 116 
uniqueness, 103—104, 116 
using strong induction, 356-359 
vacuous, 88, 116 
Proofs, by mathematical induction, 
335-350 
of divisibility results, 342—344 
examples of, 336-350 
guidelines for, 335-336 
of inequalities, 340—342 
mistaken, 349-350 
of results about algorithms, 
346-347 
of results about sets, 344-346 
summation formulae, 336-340 
Proof strategy, 89-90, 104-106 
Proper subgraph, 697 
Proper subset, 126, 195 
Properties 
Archimedean, A-5 
completeness, 944 
relations, 602—605 
of trees, 788-791 
well-ordering, 361-362, 
399, A-5 
Propositional calculus, 3 
Propositional equivalences, 
26-37, 136 
Propositional function, 41, 42, 115 
Propositional logic, 1-13, 40 
applications of, 17—23 
argument in, 74 
rules of inference for, 74—77 
valid arguments in, 73—74 
well-formed formulae in, 371 
Propositional variables, 2, 115 
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Propositions, 2—6 
atomic, 3 
compound. See Compound 
propositions 
conjunction of, 4 
definition of, 2, 85, 115 
disjunction of, 4—5 
negation of, 3, 4 
truth value of, 2, 3 
Protein interaction graphs, 680, 681 
Protocols 
cryptographic, 319-320 
Diffie-Hellman key agreement, 319 
key exchange, 324 
Pseudocode, 202-203, A-11 
Pseudographs, 675, 676, 771 
Pseudoprimes, 298—300, 324 
to base b, 299 
Pseudorandom numbers, 305-306 
generated by linear congruential 
method, 305 
middle-square generator, 309 
power generator for, 309 
pure multiplicative generator, 306 
Public key cryptography, 310, 
314-315 
Public key cryptosystems, 315 
Public key encryption, 324 
Pumping lemma, 926 
Pure multiplicative generator, 306 
Pushdown automaton, 925 
Puzzles 
Birthday Problem, 486—487 
Icosian, 734 
jigsaw, 363 
logic, 20-22 
Monty Hall Three Door, 474—475 
muddy children, 21 
Reve’s, 531 
Sudoku, 35-37 
Tower of Hanoi, 529-532 
P versus NP problem, 239-240 
Pythagorean triples, 111 


Quadratic mean, 114 
Quadratic nonresidue, 302 
Quadratic residue, 302 
Quality control, 489 
Quantification, 43 
existential, 43, 45, 116 
and looping, 47, 61—62 


null, 59 
universal, 43, 44, 49—50, 116 


Quantified expression, negation of, 


49-51 
Quantified statements 
restricted domain, 131 
rules of inference for, 79—82 
Quantifiers, 43—46 
De Morgan’s laws for, 50-51 
existential, 45—46 
finite domain, 47—56 
logical equivalences, 49 
nested. See Nested quantifiers 
order of, 62—64 
precedence of, 48 
with restricted domains, 48 
scope of, 48 
in system specifications, 53 
truth sets and, 131 
universal, 44—45 
Quasi-ordering, 669 
Quick sort, 392 
Quine, Willard Van Orman, 875 


Quine—McCluskey method, 865, 866, 


873-877 
Quotient, 253, A-6 
Quotient automaton, 916 


Radius, of graphs, 777 
Ramanujan, Srinivasa, 102, 104 
Ramsey, Frank Plumpton, 425 
Ramsey number, 426 
Ramsey theory, 425—426 
Random variables, 478, 479, 520 
conditional expectation of, 518 
covariance of, 520 
distribution of, 486, 520 
expected values of, 503-506 
independent, 511-513, 520 
indicator, 518 
standard deviation of, 513 
variance of, 503-506 
Range, of function, 147, 196 
Ratio, common, 166 
Rational numbers, 89, 116, 122 
r-combinations, 431, 450, 461 
Reachable vertex, 726, 845 
Real numbers, 122 
axioms for, A-1—A—2 
positive, 123 
upper bound, A-2 


Real-valued functions, 149 
Reasoning 
backward, 105 
circular, 94, 116 
deductive, 332 
forward, 105 
inductive, 332 
probabilistic, 474-475 
Recognizes, by finite-state 
machine, 901 
Records, databases, 612 
Recurrence relations, 167—169, 592 
algorithms and, 533-536 
applications of, 527-536 
associated homogenous, 547 
definition of, 196 
divide-and-conquer, 554-561 
linear, solving, 540-550 
linear homogeneous, 541-547 
linear nonhomogeneous, 547-550 
modeling with, 528-533 
simultaneous, 552 
solution of, 167, 527 
Recursion, 365, 386-387 
Recursive algorithms, 381-391, 399 
binary search, 384-385 
for computing a”, 382 
for computing greatest common 
divisor, 383 
for computing n!, 382 
correctness of, 385-386 
for Fibonacci numbers, 386 
linear search, 384 
for modular exponentiation, 384 
trace of, 382 
Recursive definitions, 331, 366 
of extended binary trees, 373 
of functions, 366-370, 399 
of sets, 370-374, 399 
of strings, 370 
of structures, 370—374 
Recursive merge sort, 389 
Recursive modular 
exponentiation, 384 
Recursive step, 370 
Reentrant knight’s tour, 742 
Refinement, of partitions, 649 
Reflexive closures, 628-629 
Reflexive relation, 602, 605, 665 
Regions, of planar representation 
graphs, 755, 772 


Regular expressions, 917, 938 
Regular grammars, 889, 938 
Regular graph, 702, 771 
Regular language, 889 
Regular sets, 917, 918, 921-923, 938 
Relational data model, 612, 665 
Relations, 665 
antisymmetric, 603-604, 665 
asymmetric, 609 
binary, 599 
circular, 667 
closures of. See Closures, of 
relations 
combining, 606—608 
complementary, 609 
composite of, 606—607 
connectivity, 631, 665 
covering, 655 
databases and, 612-613 
diagonal, 628 
divisibility, 650 
equivalence, 638-646, 665 
extension of, 612 
functions as, 600—601 
“greater than or equal,” 650 
inverse, 609 
irreflexive, 609 
n-ary. See n-ary relations 
parent, 607—608 
properties of, 602—605 
recurrence. See Recurrence 
relations 
reflexive, 602, 605, 665 
representing. See Relations, 
representation of 
on set, 601-602 
of sets, 130 
symmetric, 603—604, 665 
transitive, 604—605, 608, 665 
Relations, representation of, 621—626 
using digraphs, 624—626 
using matrices, 621—624 
Relatively prime integers, 281, 324 
Remainder, 253 
Rencontres, 589 
Repetition 
combinations with, 446-450 
permutations with, 446 
Replacement 
sampling with, 473 
sampling without, 473 


Representative, of equivalence 
class, 641 
Resolution (rules of inference), 
76, 78 
Resolvent, 78 
Restricted domains, quantifiers with, 
48, 131 
Results, 85 
return statement, A-15 
Reverse-delete algorithm, 840 
Reverse Polish notation, 817, 841 
Reve’s puzzle, 531 
Right child, of vertex, 785 
Right triominoes, 110, 349 
Ring topology, 695 
Rivest, Ronald, 315 
RNA (ribonucleic acid), 409 
Rocket-powered Frisbee, 848 
Root, 372, 783 
Rooted Fibonacci trees, 793 
Rooted spanning tree, 834 
Rooted trees, 372, 783-785 
decision trees, 796-798 
definition of, 783, 841 
height of, 789, 841 
ordered, 785, 841 
Roster method, 122, 195 
Round-robin tournament, 362, 
493, 692 
Routing transit number (RTN), 327 
Roy, Bernard, 634 
Roy—Warshall algorithm, 634—637 
r-permutation, 429, 450, 461 
RSA cryptosystem, 310, 324 
RSA system, 315 
cryptosystem, 315-316 
decryption, 317-318 
digital signatures, 320 
encryption, 316-317 
as public key system, 318-319 
Rule 
division, 414-415, 462 
product, 406, 462 
sum, 406, 409 
Rules of inference, 73—82, 116 
addition, 76 
and argument building, 77—78 
conjunction, 76 
disjunctive syllogism, 76 
hypothetical syllogism, 76 
modus ponens, 75, 76 
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modus tollens, 76 
for program correctness, 394 
for propositional logic, 74-77 
for quantified statements, 79-81, 
79-82 

resolution, 76, 78 
simplification, 76 

Rules of syntax, 885-886 

Run, 518 

Russell, Bertrand, 125 

Russell’s paradox, 133, 186 


Same order, big-O relationships, 219 
Same parity, 86 
Sample space, of experiment, 
470, 520 
Sampling without replacement, 473 
Sampling with replacement, 473 
Satisfiability, 33 
applications of, 33—37 
problems, solving, 37 
Satisfiable compound propositions, 
33, 115 
Saturated hydrocarbons, trees 
modeling, 786 
Scheduling problems 
final exams, 766-767 
greedy algorithm for, 346-347 
Schroder—Bernstein theorem, 
184-185 
Scope, of quantifiers, 48 
Search algorithms, 204—206, 244 
binary, 205-206, 245 
breadth-first, 826-828 
depth-first, 823—826 
linear, 204, 245 
ternary, 214 
Search engines, 831—832 
Searches 
Boolean, 19-20 
Web page, 19-20 
Searching problems, 204 
Second principle of mathematical 
induction, 355 
Seeds, 832 
for linear congruential 
method, 305 
Sees, 770 
Selection operator, 613, 665 
Selection sort, 215 
Self-complementary graph, 712 
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Self-converse directed graph, 775 
Self-dual Boolean function, 880 
Self-generating sequences, 403 
Semantic networks, 681—682 
Semantic relation, 681 
Semantics, 885 
Sentence, 886, 887 
Sentential variables, 2 
Separating set, 718 
Sequences, 121, 165-172, 398 
definition of, 165, 196 
Fibonacci, 167-168 
finding maximum and minimum 
of, 554 
generating functions for. See 
Generating functions 
Golomb’s self-generating, 403 
graphic, 701 
integer, 170-172 
Lucas, 171 
notation, 165 
recurrence relations. See 
Recurrence relations 
self-generating, 403 
strictly decreasing, 425 
strictly increasing, 424 
unimodal, 595 
Sequential search algorithm, 204 
Serial algorithms, 695 
Series 
geometric, 174 
harmonic, 342 
infinite, 176-177 
power, 564—567 
Set builder notation, 122, 195 
Set equality, 195 
Set identities, 136—139 
absorption laws for, 136 
associative laws for, 136 
commutative laws for, 136 
complementation law for, 136 
complement laws for, 136 
De Morgan’s laws for, 136 
distributive laws for, 136 
domination laws for, 136 
idempotent laws for, 136 
identity laws for, 136 
methods of proving, 139 
Sets, 121-131, 195 
atomic, 137 


cardinality of, 127, 179-186, 195 


Cartesian products of, 128—130 

complement of, 135, 136, 196 

of complex numbers, 123 

computer representation of, 141-142 

countable, 180—183, 196 

definition of, 122 

difference of, 135, 136, 196 

disjoint, 134 

elements/members of, 122 

empty, 124, 195 

equality of, 123, 195 

finite. See Finite sets 

fuzzy, 146 

guarding, 770 

identities. See Set identities 

infinite, 121, 127, 195 

of integers, 122 

intersection of, 134, 139-141, 196 

language of, 121-122 

linearly ordered, 651 

naive theory, 124 

of natural numbers, 121 

not recognized by finite-state 
automaton, 923—924 

null, 124, 195 

operations, 133—144 

partially ordered. See Partially 
ordered set 

partition of, 643-646 

power, 128, 196 

product rule for, 462 

recognization, Turing machine 
and, 929-930 

recursive/inductive definitions of, 
370-374, 399 

regular, 917, 918, 921-923, 938 

relation of, 130 

relation on, 601-602 

roster method, 122 

separating, 718 

singleton, 124 

size of, 127 

of strings, 904—905 

successor of, 146 

totally ordered, 651 

uncountable, 180, 183—186, 196 

union of, 133-134, 139-141, 196 

universal, 124, 195 

well-ordered, 402, 651, 666 


Shaker sort, 246 
Shamir, Adi, 315 


Shannon, Claude Elwood, 848 
Sheffer, Henry Maurice, 37 
Sheffer stroke, 37 
Shift ciphers, 311, 312, 313, 324 
Shortest-path algorithm, 744-749 
Shortest-path problems, 
743-751, 772 
Siblings, of vertex, 783, 841 
Sides, of polygon, 359 
Sieve of Eratosthenes, 273—278, 324, 
586-587, 592 
Signatures, digital, 320, 324 
Signed integer, 894 
Signless Stirling numbers of the first 
kind, 453, 465 
Simple directed graph, 675, 
676, 771 
Simple graphs, 674, 676, 
688-689, 771 
coloring of, 763 
crossing number of, 761 
dense, 705 
isomorphic graphs, 706-710 
orientation of, 776 
paths in, 715 
self-complementary, 712 
with spanning trees, 821—823 
sparse, 705, 711, 839 
thickness of, 762 
union of, 698-699 
vertices of, 698, 703 
Simple path, 714, 771 
Simple polygon, 359 
Simplification (rules of inference), 76 
Single-elimination tournament, 682 
Singleton set, 124 
Sink, 940 
Size, of sets, 127 
S,-tree, 843 
Slackness, of job, 247 
Sloane, Neil, 171 
Smullyan, Raymond, 20-21 
Social networks, 676 
Sollin’s algorithm, 840 
Solution, 33 
of recurrence relations, 
167, 527 
Solvable problem, 239, 245, 


933-934, 938 
Sorting 


definition of, 206, 244 
topological, 659-661, 666 


Sorting algorithms, 206-208 
bidirectional bubble sort, 246 
bubble sort, 207—208, 245 
insertion sort, 208, 245 
merge sort, 388-391, 399, 

554-555 
quick sort, 392 
selection sort, 215 
shaker sort, 246 

Space complexity, 231, 245 

Spam, 497-501 

Spanning forest, of graph, 834 

Spanning trees, 821-832, 841 
breadth-first search and, 826-828 
definition of, 821 
degree-constrained, 844 
depth-first search and, 823—826 
in IP multicasting, 823 
maximum, 840 
minimum, 835-839, 841 
rooted, 834 

Sparse graphs, 705, 711, 839 

Sparse matrix, 705 

Specifications system, 18—19 

SQL (Structured Query 

Language), 616 

Squarefree integer, 591 

Square matrix, 188, 191 

Stable matching, 216 

Standard checkerboard, 108 

Standard deviation, 513, 520 

Star height, 940 

Star topology, 695 

Start state, 905 

State diagram, 899 

Statements, A-1 1 
assignment, A-11-A-12 
biconditional, 10, 29, 115 
block of, A-14—A-15 
conditional. See Conditional 

statements 
if then, 42, 394-396, A-12—A-13 
involving nested quantifiers, 61—62 
procedure, A—11 
return, A-15 

States, 898, 905 

State table, 898 

Stirling, James, 160 

Stirling numbers of the second 

kind, 453 

Stirling’s formula, 160 


St. Petersburg paradox, 523 
Straight triomino, 110 
Strategy 
minmax, 803, 841 
proofs, 89-90, 104-108 
Strictly decreasing functions, 151 
Strictly decreasing sequence, 425 
Strictly increasing functions, 151 
Strictly increasing sequence, 424 
String matching, 209-210 
Strings, 166, 196 
bit, 12 
counting, 408 
distinguishable, 927 
empty, 166, 196, 887 
indistinguishable, 927 
length of, 166, 371 
lexicographic ordering of, 652 
null, 887 
pattern, 209 
recognized/accepted, 906 
set of, 904-905 
ternary, 537 
String searching, 244 
Stroke, Sheffer, 37 
Strong association rules, 619 
Strong induction, 354-361, 399 
in computational geometry, 
359-361 
and infinite ladder, 355-356 
proofs using, 356-359 
validity of, 354-355 
Strongly connected components 
of directed graph, 721-722, 772 
of web graph, 722 
Strongly connected graphs, 721 
Strong pseudoprime to base b, 302 
Structural induction, 366, 
374-377, 399 
examples of proofs using, 375-377 
Structured Query Language 
(SQL), 616 
Structures, recursive/inductive 
definitions of, 370—374 
Subgraph, 697, 771 
Subgraph induced, 697 
Subsequence, 424 
Subsets, 125-127, 195 
counting number, of finite set, 
344-346, 408 
image of, 149 
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inverse image of, 163 
proper, 126, 195 
sums of, 829-830 
Subtraction rule, 412—414, 462 
Subtractors 
full, 864 
half, 864 
Subtree, 784, 841 
Success, Bernoulli trial, 484 
Successor, of sets, 146 
Sudoku, 35-37 
Suitees, 216 
Suitors, 216 
Summations, 172—177 
formulae, 176, 336-340 
index of, 172-173 
lower limit, 173 
notation, 172 
upper limit, 173 
Sum-of-products expansions, 
855-856, 879 
Sum rule, 406, 409, 462 
Sums 
Boolean, 849, 879 
of geometric progressions, 
339-340 
of matrices, 189 
of multisets, 143 
of subsets, 829-830 
Sun-Tsu, 293 
Superset, 125 
Support, 618, 665 
of association rule, 618-619, 665 
Surjection, 151, 196 
Surjective functions, 151 
Symbolic Logic, 126 
Symbols, 887 
Symmetric closure, 629 
Symmetric difference, 145, 196 
Symmetric matrix, 191, 196 
Symmetric relations, 603-604, 665 
Syntax, 885 
Syracuse problem, 112 
System of distinct representatives 
(SDR), 774 
System specifications, 18—19 
quantifiers in, 53 


Tables, 612 
state, 898 
Tao, Terence, 278 
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Tautology, 26, 27, 37, 75, 115 
Telephone calls, 678 
Telephone numbering plan, 407—408 
Terminals, 887 
Terminal vertex, 625 
Ternary search algorithm, 214 
Ternary string, 537 
Test/testing 
Miller’s, 302 
probabilistic primality, 489-490 
Text, 209 
Theorems 
art gallery, 770 
Bayes’, 494-501, 521 
Bézout’s, 285-286, 324 
binomial, 437-440, 462 
Chinese remainder, 293-295, 325 
Cook-Levin, 240 
definition of, 85, 116 
Dirac’s, 736 
extended binomial, 567 
Fermat’s last, 111—112, 219 
Fermat’s little, 297-298, 325 
four color, 763—766, 772 
fundamental theorem of 
arithmetic, 272, 325, 357 
Hall’s marriage, 693-694, 772 
handshaking, 686-687, 772 
Kleene’s, 919-921, 939 
Kuratowski’s, 758—760, 772 
master, 558 
methods of proof, 86 
multinomial, 457 
Ore’s, 736 
Pick’s, 363 
prime number, 276-277 
Schréder-Bernstein, 184-185 
stating, 85—86 
Wilson’s, 301 
Thickness, of simple graphs, 762 
Threshold function, 881 
Threshold gate, 881 
Threshold value, 881 
Tic-tac-toe, 802-803 
Tilings, of checkerboards, 108-111, 
348-349 
Time complexity, 231-235, 244 
average-case, 233-234, 245 
worst-case, 232—233, 245 
Top-down parsing, 891 
Topological sorting, 659-661, 666 


Topology 
ring, 695 
star, 695 
Total expectation, law of, 518 
Total function, 161 
Total length field, 420 
Totally ordered sets, 651, 666 
Total ordering, 651, 666 
compatible, 660, 666 
Tournament 
round-robin, 362, 493, 682 
single-elimination, 682 
Tournament sort, 806 
Tower of Hanoi, 529-532 
Trace, algorithm, 203, 382 
Tractable problem, 239, 245 
Tractable problems, 936 
Trail, 714 
Transaction, 617 
Transformations, 147. See also 
Functions 
affine, 312 
Transition function, 898, 905, 906 
Transition rules, 928 
Transitive closures, 628, 630—634 
computing procedure, 634 
Transitive relations, 604—605, 
608, 665 
Transitivity law, A-2 
Translation 
English sentences to logical 
expressions, 17-18, 51-53, 
66-67 
mathematical statements to logical 
expressions, 64-65 
nested quantifiers into English, 65 
Transpose, of matrix, 190-191, 196 
Transposition cipher, 313 
Transposition error, 308 
Traveling salesperson problem 
(TSP), 737, 743, 
749-751, 772 
Traversal algorithms. See Tree 
traversal 
Tree-connected network, 787—788 
Tree diagrams, 415-416, 461 
Tree edges, 825 
Trees, 781-839 
applications of, 793-805 
binary. See Binary trees 
binary search, 793-796, 841 


caterpillar, 844 
decision, 796-798, 841 
definition of, 781, 782, 841 
derivation/parse, 890-892, 938 
family, 781 
game, 800-805, 841 
graceful, 844 
labeled, 793 
m-ary, 784 
as models, 785-788 
properties of, 788-791 
rooted. See Rooted trees 
rooted Fibonacci, 793 
spanning. See Spanning trees 
terminology for, 783 
value of, 803 
Tree traversal, 841 
inorder, 809, 811-812, 
815-816, 841 
postorder, 809, 812-814, 
815-816, 841 
preorder, 809-810, 814, 
815-816, 841 
universal address system, 808—809 
Trial division, 272—273 
Triangle inequality, 113, 146 
Triangulations, of polygon, 360 
Trichotomy law, A—2 
Triomino 
right, 110, 349 
straight, 110 
Trivial proof, 89, 116 
Truth set, of predicate, 131 
Truth tables 
for biconditional statement, 10 
of compound propositions, 11 
for conditional statements, 6 
for conjunction, 4 
definition of, 115 
for disjunction, 4 
for exclusive or, 6 
for negation, 4 
Truth value, 2, 3, 115 
Tukey, John Wilder, 12 
Turing, Alan Mathison, 924 
Turing machine, 885, 924, 925, 
927-936, 938 
computing functions with, 931 
definition of, 927—929 
nondeterministic, 931—932, 938 
representation of, 928 


and sets recognization, 929-930 
transition rules, 928 
types of, 931-932 
universal, 932 
Twin primes conjecture, 279-280 
Two children problem, 523 
Two’s complement, 270 
Type (datatype), 123 
Type 0 grammar, 889, 890, 938 
Type | grammar, 889, 890, 938 
Type 2 grammar, 889, 890, 938 
Type 3 grammar, 889, 890, 938 


Ulam numbers, 198 
lam’s problem, 112 
nary representations, of 
integers, 931 
Uncomputable functions, 185, 196, 
934, 938 
Uncountable sets, 180, 183—186, 196 
Undecidable problem, 934 
ndefined values, of partial 
function, 161 
nderlying undirected graph, 688 
ndirected edge, 677, 771 
Undirected graphs, 675, 676, 685, 
687, 771 
connectedness in, 716-718 
density of, 711 
Euler circuit of, 729 
orientation of, 776 
underlying, 688 
Unicasting, 823 
Unicycle, 848 
Uniform distribution, 479, 520 
Unimodal sequence, 595 
Union 
of finite sets, 579 
of fuzzy sets, 146 
of graphs, 698-699, 771 
of multisets, 143 
of sets, 133-134, 139-141, 196 
niqueness proof, 103—104, 116 
nit-delay machine, 900 
nit (Egyptian) fraction, 401 
nit property, in Boolean 
algebra, 851 
niversal address system, 808—809 
niversal generalization, 79-80 
niversal instantiation, 79, 80 
niversal modus ponens, 81 
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niversal modus tollens, 81—82 

niversal Product Code (UPC), 
306-307 

Universal quantification, 43, 44, 

49-50, 116 

Universal quantifier, 44—45 

Universal set, 124, 195 

degree of membership, 146 

niversal transitivity, 84 

niversal Turing machine, 932 

niverse of discourse, 44, 115 

nmatched edges, 693 

Unsatisfiable compound 
propositions, 33 

nsolvable problem, 239, 245, 
934, 938 

UPC check digit, 306-307 

Jpper bound, 666 

of lattice, 669 

least, 657, 666, A-2 

of partially ordered set, 657 

of real numbers, A—2 

Upper limit, of summation, 173 

Upper triangular matrix, 244 

U.S. Coast Survey, 42 
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Vacuous proof, 88, 116 
Valid argument, 73-74, 116 
Valid argument form, 74, 116 
Valid partner, 247 
Values 
of Boolean function, 849 
expected, 503-506 
final, A-14 
initial, A-14 
threshold, 881 
of trees, 803 
truth, 2,3, 115 
of vertex in game tree, 803-805, 841 
Vandermonde, Alexandre-Théophile, 
442-443 
Vandermonde’s identity, 442-443 
Variables, A—14 
binding, 48—49, 116 
Boolean, 12, 115, 848, 856, 879 
free, 48, 116 
propositional, 2, 115 
random. See Random variables 
sentential, 2 
Variance, 503, 513-516, 520 
Veitch, E. W., 866 
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Vending machine, 897—898 

Venn, John, 126 

Venn diagrams, 124-125, 195 
complement of sets, 136 
difference of sets, 136 
intersection of sets, 134 
union of sets, 134 

Verb, 886, 887 

Verb phrase, 886, 887 

Vertex (vertices), 625 
adjacent, 685 
ancestors of, 783, 841 
bipartition of, 689 
child of, 783, 841 
in collaboration graph, 677 
connecting, 685 
connectivity, 718-719, 720-721 
counting in full m -ary trees, 

788-789 

counting paths between, 723-724 
cut, 718, 771 
degree, in undirected graph, 685 
descendants of, 783, 841 
distance between, 777 
dominating set of, 775 
eccentricity of, 793 
handle, 843 
in-degree of, 687, 771 
independent, 777 
in influence graph, 677 
initial, 625 
interior, 634 
internal, 783, 841 
isolated, 686, 771 
left child, 785 
left subtree, 785 
level of, 789, 841 
level order of, 843 
mutually reachable, 726 
out-degree of, 687, 771 
parent of, 783, 841 
pendant, 686, 771 
of polygon, 359 
reachable, 726, 845 
removing from graph, 698 
right child, 785 
right subtree, 785 
siblings of, 783, 841 
of simple graphs, 698, 703 
terminal, 625 
value in game tree, 803-805, 841 


I-26 Index 


Vertex basis, 727 
Vertex cut, 718 
Vigenère cipher, 322 
Vocabulary, 887, 938 


Walk, 714 
Warshall, Stephen, 634, 636 
Warshall’s algorithm, 634-637 
Weakly connected graphs, 721 
Web crawlers, 831-832 
Web graphs, 678 
strongly connected components 
of, 722 
Web pages, searching, 19-20 
Weighted graphs, 772 
shortest path between, 743-751 
Weights, 881 
Well defined, 367 
Well-formed formulae, 
371-373, 821 


Well-founded partially ordered 
set, 664 


Well-ordered induction, principle of, 


652 
Well-ordered set, 402, 651, 666 
Well-ordering property, 361-362, 
399, A-5 
Wheels, 689, 771 
Wiles, Andrew, 112 
Wilson’s theorem, 301 


Without loss of generality (WLOG), 


100, 116 
Witnesses 
big-O notation, 217, 244 
and existence proof, 101 
WLOG (without loss of generality), 
100, 116 
Word, 887 
Word and Object, 875 
Worst-case complexity, 
232-233, 245 


of binary search, 232-233 
of bubble sort, 234—235 
of insertion sort, 234—235 


XOR gate, 881 
XOR operator, 13 


Zermelo-Fraenkel axioms, 186 
Zero—one matrix, 191-193, 196 
Boolean product of, 192-193, 
196, 236 
join of, 192, 196 
meet of, 192, 196 
representing relations using, 
621-624 
of transitive closure, 633-634 
Warshall’s algorithm and, 
634-636 
Zero property, in Boolean 
algebra, 851 
Zhang, Yitang, 280 
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TOPIC SYMBOL MEANING PAGE 

LOGIC =p negation of p 3 
DAG conjunction of p and q 4 
DV disjunction of p and q 4 
pq exclusive or of p and q 5 
p>q implication p implies q 6 
peq biconditional of p and q 10 
p=q equivalence of p and q 27 
T tautology 29 
F contradiction 29 
PO 52863%_) propositional function 42 
VxP(x) universal quantification of P(x) 44 
AxP(x) existential quantification of P(x) 45 
A!xP(x) uniqueness quantification of P(x) 46 
s therefore 73 
P{S}q partial correctness of $S 393 

SETS xES xis a member of S 122 
xésS xis not a member of S 122 
{ai ..., an} list of elements of a set 122 
{x | P(x)} set builder notation 122 
N set of natural numbers 122 
Z set of integers 122 
Zt set of positive integers 122 
Q set of rational numbers 122 
R set of real numbers 122 
[a, b], (a, b) closed, open intervals 123 
S=T set equality 123 
ø empty (or null) set 124 
SCT S is a subset of T 125 
SCT S is a proper subset of T 126 
|S| cardinality of S 127 
P(S) power set of S$ 128 
(ay, ..., a,) n-tuple 128 
(a, b) ordered pair 128 
AXB Cartesian product of A and B 129 
AUB union of A and B 133 
ANB intersection of A and B 134 
A-B difference of A and B 135 
A complement of A 135 
(Ja; union of A, i= 1,2, ...,n 140 
| 
(Ai intersection of A, i = 1, 2, ..., n 140 
i=l 
A@B symmetric difference of A and B 145 
No cardinality of a countable set 180 
c cardinality of R 185 


TOPIC SYMBOL MEANING PAGE 
FUNCTIONS fía) value of function f at a 147 
fAa-B function from A to B 147 
fith sum of functions f, and fọ 149 
fih product of functions f} and f 149 
F(S) image of set S under function f 149 
14(S) identity function on A 153 
F inverse of f 153 
fog composition of f and g 155 
|x] floor function of x 157 
[x] ceiling function of x 157 
an term of {a;} with subscript n 165 
> 4; sum of a), >, ..., a, 172 
i=l 
È a, sum of a, overa E€ S 175 
aes 
IIa, product of aj, az, ...,d, 179 
i=l 
f(x) is O(g(x)) f(x) is big-O of g(x) 217 
n! n factorial 160 
f() is Q(g(x)) f(x) is big-Omega of g(x) 227 
f(x) is O(g(x)) f(x) is big-Theta of g(x) 227 
~ asymptotic to 231 
min (x, y) minimum of x and y 281 
max (x, y) maximum of x and y 282 
z approximately equal to 472 
INTEGERS a|b a divides b 252 
afb a does not divide b 252 
a div b quotient when a is divided by b 253 
a mod b remainder when a is divided by b 253 
a = b (mod m) a is congruent to b modulo m 254 
a £ b (mod m) a is not congruent to b modulo m 254 
Zin integers modulo m 257 
(AjAp_1 ++» 4A)» base b representation 260 
gcd(a, b) greatest common divisor of a and b 280 
Icm(a, b) least common multiple of a and b 282 
MATRICES la; matrix with entries aj 188 
A+B matrix sum of A and B 189 
AB matrix product of A and B 189 
I, identity matrix of order n 190 
A’ transpose of A 191 
AVB join of A and B 192 
AAB meet of A and B 192 
AOB Boolean product of A and B 192 
Al] nth Boolean power of A 193 
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TOPIC SYMBOL MEANING PAGE 
COUNTING P(n, r) number of r-permutations of a set 
AND with n elements 429 
PROBABILITY C(n, r) number of r-combinations of a set 
with n elements 431 
(") binomial coefficient n choose r 431 
C(n; ni, Ns, ...,,,) multinomial coefficient 457 
D(E) probability of E 470 
DE | F) conditional probability of E given F 481 
E(X) expected value of random variable X 503 
V(X) variance of random variable X 513 
C, Catalan number 533 
NP; e Ph) number of elements having all the properties 
Ppj=1,..Ţṣn 585 
N (P; sP) number of elements having none of the properties 
" Bjal 585 
D, number of derangements of n objects 589 
RELATIONS SoR composite of relations R and S 606 
R” nth power of relation R 607 
R! inverse relation 609 
Sc selection operator for condition C 613 
an ae projection 614 
J,(R, S) join 615 
A diagonal relation 628 
R* connectivity relation of R 631 
a~b a is equivalent to b 639 
lalp equivalence class of a with respect to R 641 
La] m congruence class modulo m 642 
(S, R) poset consisting of set S and partial 
ordering R 650 
a<b a is less than b 651 
a>b a is greater than b 651 
agb a is less than or equal to b 651 
axb a is greater than or equal to b 651 
GRAPHS (u, v) directed edge 625 
AND TREES G=(V,E) graph with vertex set V and edge set E 673 
{u, v} undirected edge 674 
deg(v) degree of vertex v 685 
deg-(v) in-degree of vertex v 687 
deg*(v) out-degree of vertex v 687 
K, complete graph on n vertices 688 
C, cycle of size n 688 
W, wheel of size n 689 
Qn n-cube 689 
Kmsn complete bipartite graph of size m, n 691 
G-e subgraph of G with edge e removed 697 
Gre graph produced by adding edge e to graph G 697 


TOPIC SYMBOL MEANING PAGE 
GRAPHS AND G UG, union of G, and G, 699 
TREES (cont.) a, Xi, ...,X,_1,0 path from a to b 714 
A,X, ...,Xp_4,4 circuit 714 
K(G) vertex connectivity of G 718 
A(G) edge connectivity of G 720 
r number of regions of the plane 756 
deg(R) degree of region R 757 
x(G) chromatic number of G 763 
m greatest number of children of an internal 
vertex in a rooted tree 784 
n number of vertices of a rooted tree 788 
i number of internal vertices of a rooted tree 788 
l number of leaves of a rooted tree 789 
h height of a rooted tree 790 
BOOLEAN x complement of Boolean variable x 847 
ALGEBRA x+y Boolean sum of x and y 847 
x- y (or xy) Boolean product of x and y 847 
B {0,1} 848 
F’! dual of F 852 
x|y x NAND y 857 
xly x NOR y 857 
x E > inverter 22, 859 
x 
F >e **Y OR gate 22, 859 
x 
a DS a AND gate 22, 859 
LANGUAGES A empty string 166, 887 
AND xy concatenation of x and y 371 
FINITE-STATE I(x) length of string x 371 
MACHINES wk reversal of w 380 
(V, T, S, P) phrase-structure grammar 887 
S start symbol 887 
w>w production 887 
wi >w, w, is directly derivable from w}. 887 
w Š We w, is derivable from w4. 887 
(A) ::= (B)c | d Backus—Naur form 893 
(S, L O, f, 8, So) finite-state machine with output 898 
So initial or start state 898 
AB concatenation of sets A and B 904 
A* Kleene closure of A 905 
(S, L f So E) finite-state machine automaton with no output 905 
(S, LF So) Turing machine 927 


